History's Worst Software Error 

8 billion people on the planet, they wills say. It's not a big loss if one person dies, but affect the company and many more people will be drastically affected when we lay them off, they will say.

And nothing has changed.

What do you mean will? All of them just do, that is their business model. For most companies if they could just feed their employees and customers into a meat grinder and be rendered the profit expected from that person over their lifetime, that's what the company would be doing

I don't know, this screams incompetence to me on every level more than malice.

The company in question is wholly owned by the Canadian government.

But why does it fail? Worked yesterday but doesn't work today, what has occurred?

@@SebAnders what’s crazy is electronics are so sensitive literally radiation from space can fuck up your data. You could do everything correct, program your software perfectly, use the hardware as intended and then bam: universe said no you will get error

@@SebAnders software is more finicky than a nervous system made entirely out of worms. Sometimes I beg my computer in a sweet voice to obey because I have about the same chance in that working as anything else. Software and computers generally can be really finicky, depending on what they are.

Totally wrecked my immersion lmao, whoever wrote it obviously hasn't even poked html.

@@SebAnders black magic

Ikr? Medical gaslight is absurd. If a patient reports a damage, the staff must register and report it! I work at an hospital and, unfortunately, some people think that they'll be punished if the report such a thing, when there are literal peoples' lives at risk!

Crazy?

it’s actually disgusting how much medical gaslighting happens

​@@Dan-fc1hpI was crazy once

"hey my chest is burning with the force of a thousand suns and all my skin is peeling off." "nah you buggin."

as soon as it said MALFUNCTION, it should have been reset and restarted, all would have been gone well

You'd be surprised how many completely benign errors that critical equipment throws - they *should* have formally troubleshooted them but I wouldn't want to be too harsh on a group of not-actually-very-well-paid technicians who were likely under substantial time pressure and had been given *no* training on how to troubleshoot these devices, or even a reference as to what the errors mean.

​@@bosstowndynamics5488yup, management is more concerned with the bottom line, and having cash-producing equipment sitting idle is the last thing they'd want. Safety they say is paramount... until it's not.

You have NO idea how the technician-level people work, do you? They are just instructed what buttons to press and what to do when some error comes up - they have NO technical expertise or knowledge to know the consequences. It's not their fault, that's how business works. This is entirely the fault of the manufacturer. Any system must, must, must be idiot proof assuming that a monkey is going to operate the machine and under no situation will such a malfunction happen. God we're talking lives here!

there was only one dangerous malfunction out of 64, so the likelihood is that the operators experienced many false alarms before

and its not like it was a phone that threw up error messages. it was a machine that shoots you with fucking radiation.

Yep... as a hospital, I would at least demand a list of the meanings of all error codes before using such a machine again.

That's the common sense part, and we could also call it empathy, integrity, self-respect. I have seen this problem many times. There are psychological studies about it, sociological studies, about authority, fear, all that. We don't seem to learn, only adapt/comply temporarily through coersion, then repeating the folly, but with even more deadly tools developed by now.

Doctors did ask for meanings behind these errors. Company never gave them. I wonder why. What was the reason they didn't want people knowing what these errors meant and also the fact that there were errors and the company knew about it in the doctor's knew about it and they just simply ignored them is a bit odd if I was going to the doctors I wouldn't want their medical equipment coming up with any errors ever and if they were I would hope that my procedure would be halted until further notice

@@ZuraTheCat just guessing, they got too many sensors and error sources to fit 64 error code. So error 54 maybe just some generic hardware error. So they think the machine will be not useable if it have to reset for such generic error

Sure... unless you make cigarettes, thalidomide, or homebrew IEDs. Then it's lie through your teeth baby, and keep those profits rolling in!

@@dexterpoindexter3583 oof. Thalidomide was good pull. That shit was literally/deliberately sold to _pregnant women_ , only for each baby to be born with fucked up limbs. All of them.

...One of those is VERY much not like the others lol

@@dexterpoindexter3583 can’t wait to buy my ied’s at the nearby gas station!

Yeah, never asked the question "Then where did that radiation come from?"

I’m sorry for your loss, my dad died in 2022 while I was in college and it’s been lonely without him

@@Shiturd45 I kind of got used to him being gone. I think the hardest part of it all was that the cancer was in his brain, and damage to his brain structures was what got him, but long before he died, he changed in ways that made him very difficult to be around. I hear stories of how he used to be a great man, but I never got to see that. Everyone else mourned someone I'd never met.

This brought me to tears. I hope you have had and will continue to have a very successful and impactful career.

@@authenticallyyou7475 Currently working with games industry veteran friends of mine who are tired of how things are done, these days, and our first game as a studio should be coming out in Q2 or Q3 of 2025, if all goes according to plan.

@@sarahfay5280I’m so sorry for your loss. That must have been so hard. I experienced something similar with my father’s confusion and irritability at the end of his fight with lung cancer. I can’t imagine not being able to remember what he used to be like.

They really were, and effects of that are still felt to this day.

Oh man. Just oh man. They are not different.

That's to. But why there was even a option to blast deadly dosages? There shouldn't be a function for that.

im sure software design was uh both undeveloped at the time as well as assummed that the people operating the machines would know to not

Reminds me that the worst alarm is one that is correct half the time. If it’s correct more that’s great, if it’s correct less that means if you do the opposite of what it says it’s still better than half right. When you’re working with something that gives you constant errors that don’t seem to do anything or have an explanation it’s easy to skip them because they’ve never done anything before. How often do you read terms of service when it pops up? Everyone’s used to clicking past them. Documentation and Testing is Everything in software ethics, what a massive failure.

You would get that call if the claims weren't true, too. And it's reasonable for companies to try and avoid PR disasters over small mistakes that happen, but of course when knowingly putting human lives at risk, that's a completely different story.

Most of the time corporations are dealing with malicious rumors spread by their competitors or activists, so it makes sense for them to start with cease-and-desist.

@@derkevevin Wrong. Non-evil would say they are investigating the matter, instead of just telling you to shut up.

@@derkevevin fair , things like that can easily be falsified and used for worse evil. Just plain blackmail even

​@@derkevevin yeah that is a very valid point -- there's no shortcut to good judgment. For every situation like this where there is a true problem, there are 100 activists crying wolf about whatever particular thing they're biased against. An unrelated example I know from a past job: nuclear power plants and the NRC have a (healthy) culture of extreme safety paranoia and publicly report every minor problem or employee failure, e.g. "this widget was supposed to be inspected every week, but was mistakenly only inspected every 2 weeks. The personnel involved have received corrective training." Anti-nuclear activist groups will read those self-published reports, and then repeat them in inflammatory language as if they've uncovered some sort of conspiracy. "DOZENS of safety issues reported!!!! Donate to us today so we can keep up this good work!"

Good news! Its failure rate has been reduced from 99.99999% to only 99%!

Reminds me of the Errol Morris interview of Training Check Airman, Denny Fitch. Fascinating story.

Math is not your strong suit. You had better avoid statistics.

@@HenriFaust L + ratio

@@HenriFaust Maybe try explaining the fallacy, rather than just insulting their intelligence.

Underrated comment

Do you think it killed more people by not killing their cancer cells than it did by radiation poison?

Bingo. As there are overflow errors that went unchecked, there were likely also underflow errors present that just didn't get noticed at the time, if it ever was before the machine became antiquated.

Lol, homeopathic radiotherapy

​@@HuyNguyen-iv3kgMore like: underirradiated comment

No, its infuriating.

Negligent, irresponsible, a dereliction of duty.

Victims were not shareholders

​​​@@Sockem1223Shareholders are the Canadian State; AECL is what we call a Crown corporation.

Don’t work in a hospital, but as a software engineer, at my job the number of times we have to tell a vendor what is wrong with their own product is extremely alarming.

and corporations, by their legal status, are never held responsible for murder or any actually serious offense. funny how that works, huh? also, their legal status prevents the individual, which in cases like this is 100% guaranteed to be top-level executives, from taking the full blame/consequence of their actions.

@@ETXAlienRobot201 If corporations are people when is the US going to give the death penalty to one?

@@ETXAlienRobot201 only in America. Look up the 2008 Chinese milk scandal. Executives got death sentences for their negligence

capitalism moment

@@VestigeFinder such a situation could also happen in communism. Imagine if this was China, and AECL was owned by a high ranking member of the CCP. Probably thousands would die and nothing would come of it.

Exactly. This is why modern medical devices have a standardized development and testing process dictated by the FDA so that you don't have to rely on the one guy.

Best thing about computers is that they do exactly what you tell them to do. Worst thing about computers is that they do exactly what you tell them to do.

The best thing about computers? They do exactly as you tell them The worst thing about computers? They do exactly as you tell them

When the team finds excuses for not writing tests, because of time constraints. 🤦🏽‍♂️

They can think the Same,but faster, and memorize more possible situations , so technicly, , they are smarter (like chess machines)

frr

That sounds fucking terrible. So sorry to hear that

I know. I know. The unwashed masses of lab rats will make real contributions to tech for the elite.

Do you know the exact or approximate date and hospital?

Talking about it? You guys kill people and call it treatment. 🇨🇦

More info is requiered!

I assume this would be very sobering in your position. I hope they take it seriously and to heart.

Thank you for your advocacy!

You potentially saved lives doing that. Congrats and please don't stop being an awesome and thoughtful human being

Thank you.

You are a good boss.

It doesn't seem that unbelievable to me. Constantly seeing medication commercials showing smiling faces while saying that death is a possible side-effect.

5 seems like a small number if you imagine that information was not wide spread as it is today. but still a big uff

Thats American doctors for ya

@@killerzer0x74 bro... this didn't just take place in america. it literally mentioned canada, among others

@@DouglasSilva-bq4xq Yeah it was the 1980's, if this happened today it would make headlines probably after the first victim

It feels weird, because when I get an error message on my registers at work, I'm there going through the diagnostics I myself can do immediately, and if those don't work, I'm on the phone with IT to see if they have any remote fixes they can do, and if those don't work, a work orders put out for it to be fixed ASAP. And that's just for a register at a regular old retail job, no lives on the line if it malfunctions or crashes in the middle of a transaction, just an inconvenience while I move the customer to another register

Ok but if you get them several times a day and when you call the pros who sold you the item tell you to just press P to proceed (which they surely did because instead of taking time to explain each malfunction, all technicians would do the same action, pressing P to proceed) it would not be so concerning after a week of it.

The immediate predecessors to the machine in question were physically incapable of delivering dangerous doses of radiation, in that context, and knowing that many of those errors were benign (the Therac 25 killed 6 people but each unit was throwing on average 4 errors every single day) I can absolutely see a tech not realising that some of those errors could be dangerous.

It was desensitization. If humans experiences something repeatedly sometimes we can get used to the most horrifying things. They were getting errors multiple times a day and usually they didn’t end up with dead people. Why would they assume it would be any different for this particular error message?

@@pappanalabI used to work hospital IT in 2018 There are still 100s of error codes in 2018 and people just hit ok or reboot the machine 1/3 of the people I helped didn’t even know there’s a computer attached to the monitor

Was going to point this out, thank you.

almost as if technology is a blessing until someone who can decides to make it a curse. it's not the technology, it's always the operator who fucks up. because even if the machine isn't running properly, it's up to the operator to fix it

@@justalittleguywithsomeproz1162 Ok, explain how the operator could "fix it" then bro. The software did not change the beam type after they had explicitly typed it in. It instead gave an obscure error code that did not exist in any manual, and lied about the beam type being changed (displaying the wrong type). You are not going to turn this around on the operators.

@vacuum sealed Garfield capitalism amiright

@@Tulanir1 You are on the same page I think, just labeling the operator differently. In this case, no it was not the doctors working with the device, it was the people greenlighting its use over and over again after it was proven unsafe.

@@GuntramEverum In this case it was mostly the fault of the company that developed it and denied the malfunctions and those who did not force them to fix it properly. The operators could be blamed for continuing to use the machine after it was found to be dangerous but they were told the machine was safe

I hate labour and safety laws

Why? Because they save lives, stop injury and stop employers screwing over their workers? Because private companies sure as hell wouldn't have come to treat employees well without being told to.

@@australiananarchist480 Then you are either a very selfish person who never has to work in dangerous situations, or you are a complete idiot.

@@australiananarchist480 If you're actually an anarchist, you have a piss-poor understanding of how and why those labor and safety laws came to be.

There are so many blood soaked laws and safety protocols in the army... I remember that pretty much every time we heard a safety something, it was accompanied by someone not doing that and someone getting killed. Remember everyone, a gun is a weapon that kills. Don't fear it, but be careful.

You forgot that you're doing software development 40 years later with much stricter safety standards

@@donothesitate1198 Mostly because of cases like this one. This happened throughout all the existing industries back then. In truth, given how many scientific and industrial development leaps were done at the time, a lot was unknown and several horrendous events happened. There was a lot to learn, but hubris an ignorance often got the better of people.

I had to do an assignment for a college class in which I had to make a vending machine for drinks, with the possibility to add from 0 to 4 ice cubes, and 3 different drinks. It was on VHDL, and we were programming an FPGA board. When me and my mate were presenting the project, my professor started asking why we implemented a button to confirm our selection, that it was not needed and "too many buttons". I explained my reasoning (it was my idea to implement it), stating that because we need to choose the drink, the ice and since the board had limited selection, that it was better to had a confirmation button so that a "costumer" would not make a mistake and end up with something they didn't want. His solution was to put a single input for beverage selection (which would mean it would have to loop around in case you wanted like the 1st drink and you were on the second). Didn't even suggest to have it starting "pouring" the drink after a set amount of time after drink selection. I genuinely was surprised how little that professor cared about something that could cause a problem that could have a simple solution. Got penalized in my grade for that as well. It is somewhat weirdly and morbidly funny how I've heard this and several other stories about how software errors have caused deaths/economic disasters in several classes for quite a while in college, just to have a professor not care about it at all.

@@Gfious Was your education at least free, or did you spend money on that experience?

@@TRAMP-oline have to pay for it. Not that much (less than 100$ montlhy), but it still is a strain.

@@Gfious I mean, honestly, they're preparing you for the real world. This incident was a company squeezing pennies hiring some hobber instead of someone qualified, and your extra bells and whistles, might please the customer, it wouldn't please who you were working for because it cost more than his solution... which is sad, but that's the reality. This practice didn't end what's in this video either; it's still going strong.

@@Gfious You should have told him that he was insisting you create the mixed drinks equivalent of the Therac 25, where a person impatient for a drink could end up with who knows what in the glass rather than what they wanted.

Yeah. There should be a law, to sue such companies. On other hand making of medical equipment is a really hard thing. Obviously a chance of being sued will increase a price of already very expensive equipment. And people will blame doctors for overpriced treatment(part of price is of course a salary of doctor).

If you have a possibility to go to jail for a mistake while making something, then the price of this thing will rocket jump in to the sky.

They didn't destroy them. They gave them a torturous end.

What do you expect? America doesn't prosecute c o m p a n i e s. Even though they can be as psychopathically heartless, deadly, and remorselessly prone to recidivism, as the country's worst serial killers.

​@@Youvko The risk of a lawsuit is already there. The company and hospital *were* sued, just not prosecuted. The argument that greater oversight and accountability, as well as increased safety protocol, upheld with legal and criminal ramifications, will lead to increased costs to the consumer, is beyond me. These are changes that can only benefit people. If instigating them puts them out of the financial reach of people, then the next changes need to be structural and aimed at the health care system itself.

The computer didn't make an error... the software developers and the machine operators did by ignoring error conditions...

He’s right ⬆️ the computer is never wrong, it’s the programmer

@@naturalvids1200 Computers can be wrong - it's just extremely rare. That's why critical systems like space missions will have 3 computers where 2 must agree. As a 40 year career programmer I myself have seen one case were a random bit-flip passed CRC checks and caused a wrong answer.

@@naturalvids1200 The computer *can* be wrong... if cosmic rays flip a bit. It's a joke amongst my software engineering team that inexplicable errors are caused by "sunspots", ie cosmic rays.

It is funny how this rings so close to, “Guns don’t kill people, people kill people” though…just an observation.

Wasn't stupid proof

It’s actually unbelievable that you can go through med school and not have any common sense.

I honestly hope all who did are haunted by what happened to these people

@@sixthsecond I less blame the operators and more blame the people who trained the technicians and the operating manuals for the device. If you face multiple error messages daily, the company tells you not to worry about them and to proceed, and nothing documents what the error messages mean or what action you should take, it's hardly the technician's fault for proceeding with their job. The company who made the software should not have even allowed technicians to bypass the error codes and proceed. Or they should have provided readable messages saying what the technician should do when the error happens. Or failing that, have a book with the error codes that can be looked up and what action should be taken. Sure it's easy to say that the technicians should have used common sense in hindsight and not ignored error codes but when nobody is telling you the severity of the errors or what you should do about them, that is the fault of the trainers from the hardware company and the documentation. There was also a failure in management. The fact that the devices continued to be used after multiple failures resulting in death is baffling to me but this again would not be up to the technicians.

@@shan8130 Technicians who operate xrays and other types of radiation devices generally don't go through medschool if that's what you mean. It's generally a 2 year college program. Technicians aren't MDs. However even if they were MDs, I'm not sure how much blame I'd assign to the operators.. The fact that so many error messages happened constantly, none of them were documented anywhere, none of them had any steps that the operator should take, and some of them were more severe than others without actually indicating which ones were serious and which weren't puts the majority of the blame for this with the company who developed it and trained technicians to use it.

To be clear, he coded a DIFFERENT device, and then they just reused the code for this machine as-is. It's even less safe than you're making it sound 🤣

The most atrocious thing here, was not his programming, but the horrible management practices of how they handled the code base. It sounds like they didn't even try to code review it, or test it beyond seeing if it "worked".

“Science”

That's quite rad.

It was a very different time in the world of computing when the only people who could afford it were wealthy hobbyists and government institutions.

We need to get this man an award for this series

Not just on RU-vid.

​“Those who cannot remember the past are condemned to repeat it.” - George Santayana, The Life of Reason, 1905.

It takes a lot of effort not to make a VALVe joke at this point.

@@lillywho I wish it was connected somehow. We'd only have 2 victims if that was the case.

what did he mean by that sentence tho god-

Most people are idiots, I approach all things with the understanding that most people involved in the process were idiots, with varying degrees of good intentions.

"99 little bugs in the code, 99 little bugs in the code, take one down, patch it around, 127 bugs in the code"

Intercom code was Fd up too!!!

thats crazy...whenever i bave MRIs they give me a little button to press in case for some reason they cant hear me.

@@sexygirlmax2019 And I reckon incidents like the Therac 25 are precisely why.

Okay can we talk about the intercom malfunction? There’s probably a 100 different things that could happen in that room that would warrant REQUIRING an intercom. I get it, stupid things happen all the time in hospitals, but I feel like a soundproof room with no way to communicate is probably not a great idea even if it’s unlikely something’s gonna go wrong.

Strongly agree.

That was exactly what I was thinking. They would have totally thrown him under the bus. 😒

@rohanorton I'm glad that they deemed it not his responsibility because in all truth it was the company's responsibility to take action and fix it. Their product not his

As someone who writes code for machinery; you always, always, always test what you wrote. And you don’t just test to see that it works, you try to break it. Then you get more people to test out the code after you think you got all the bugs out.

@@StruggleButtons yeah I totally agree. The code is made by a hobbyist. He might not even ever seen the machine. Also, it was made for an older machine and poured it to a newer machine and I doubt that they even thought about testing it, which is a shame. But so long as we remember this, hopefully we'll never repeat it and we'll learn from the past

And in recent news research supports that the sky may be blue

"it's only radiation"

💀

Here's the thing: bc it was genuinely nothing to see there. 6 instances were mentioned here (not all caused by the same malfunction) but the therac would actually throw malfunction messages daily... and nothing happened. They were told it was safe, and normally it also was. The therac normalized these error messages as nothing dangerous

@@catalayalafaye5337 exactly. And that was exactly the problem: there was ZERO distinction between harmless errors and dangerous ones.

Test-Driven Development addresses this issue pretty well. Have someone knowledgeable involved in writing the acceptance tests - and fuzz test all inputs!

@MY DOG SAYS BJÖRK actually that was due to a design change in the aircraft. The addition of new engines mounted in different positions without any update to software caused the issue.

@@JenkemSuperfan actually it was caused by deliberate intent to mislead airline operators into believing that no substantive change had been made to an aircraft variant from its baseline model, when in fact it it should have been considered a different aircraft with the unique pilot qualification and service requirements

@@JenkemSuperfan Um, no. The design change did change the aerodynamic of the craft anf software was written to compensate. That software, combined with only checking one AoA sensor, caused the crashes.

@@sarahsmith840 the software was designed to deal with an airframe operating different engines in a different position and either not enough or no adjustments were made to the sofware to compensate

It's not a "bad" field, but computers are just unpredictable in weird ways and coding is hard as shit as stuff you wouldn't imagine could happens, happens. So you need to continuously solve for bugs until your product works enough

@@mmyz7 didn't say it was a bad field, I love it, and I was just referencing an xkcd lol

@@mmyz7 That's just wrong. Computers are predictable in the best possible way. They will do exactly what you tell them, to the letter, barring rare events like bit flips from radiation. The issue, like is often the case, is human error.

Many software developers are completely clueless when it comes to machine control. There is a long history of putting a program on a bunch of punched cards, loading them on to a reader, and waiting until the computer gets around to running the program once through and creating a printout of the results, usually by the next day. This is quite different than running said program 100 times per second, as is common in machine control applications. VT100 terminals and VAX computers were only a few years removed from the height the punched card era. Vax computers were never meant for low level machine control (real time positioning and activation of physical devices). Although used extensively in industrial settings, their use is typically limited to process models and retrieving orders from the business computer, while leaving the machine control part to more specialized computers. There is a current trend to use Linux and Windows PC platforms but with a specialized OS extension designed for direct control of machinery or other physical devices.. The punched card paradigm very much resembles the current fashionable functional/stateless programming which is a disaster waiting to happen should those techniques ever be applied to machine control, IMHO.

@@richardjafrate5124 So if I can rephrase, the stupidity of the day is formatting instructions in a generalist way, making them useless for specific systems?

But it didn't end there as when the litigation started the hospital had a barrage of lawyers with the specific intent to destroy their (the victims) lives even further, digging up dirt on whatever they could find, and then making it turn into their fault, that they were the evil people.

i dont know.. sorry :(

Idk is my name T-20?

No clue, but I'm sure the T-800 has much better performance values.

@@theoddball3850: Well, sure, but there's the whole "They were all designed to hunt out and exterminate all human life" thing; if we overlook our inherent biases, then I'm sure we'd all agree with Skynet's annual performance reviews of the entire line. Well, mebbe not the _entire_ line; I heard there was one single unit that was reprogrammed to go rogue.

It's almost exactly what investigations found. i still have some of the evaluation reports on this and the medtech ethics class study on the case, but the THERAC-20 did not have radiation accidents. There was a bug in the T-20's software locks that made the code ignore the failsafes, but the hardware locks were functional. They removed the hardware checks for the THERAC-25.

also maybe have a big team to help code your nuclear device instead of some dude out of his mom's basement

Reminds me of the Bartender Robot joke, the developers made sure that the robot could make all the drinks in whatever quantity needed, the first real user asks the robot where is the toilet, the robot explodes.

@@fizzinsoda clinics and hospitals don't have enough money in their budgets for that, it is sad. We need to subsidize the medical industry, not the coal and fossil fuel ones.

@@fizzinsoda I guess that this problem was more common in those days when people had no clue about software programming and didn't take the matter serious. If you think that your computer magically does everything it should, you are not going to address software engineering problems with the caution that they should be addressed with. Luckily today most people in the tech industry have at least some basic idea of what computers do.

@@hakimmohamad6216 as a software engineer myself, no, they really don't. I left a medical device company last year because writing unit-tests were "not in the budget".

Putting two modes with different power levels is fine, in isolation. The high does was necessary for the x-ray treatment. The actual problem was more just the fact that there was a shocking lack of safety checks and failsaves and hardware interlocks. ALL of those, and more, should have been present.

@@bennowakowski3099 in isolation is doing a lot of heavy lifting here. The lethal doses should have had a physical restraint when shipping out "xray machines". Edit: For example, maybe they could have limited the power, so even in high power mode, it wouldn't produce so much. The company was too lazy (read:cheap) to either create separate machines or to modify existing ones so no harm could come to the patient accidentally. The 80s were wild.

High-end motherboards even have error codes indicators instead of just lights.

Not just that, we expect it to fail. Not sure if he ever used a computer

Totally agree..When I write code "I don't think the software as something being able to WORK" xD

In the 1980s did they even have programs to check for errors?

@@bellabear653 just some more lines of code

@@gerrypaolone6786 True but important lines that can catch critical errors. Every Coder needs a Spell check.

it seems to me the biggest change is that its now much more common to assume that software CAN do seemingly impossible problems, rather than assuming that it CAN'T.

@@kevindaniel1337 and doing those impossible problems make a lot of hard tracable errors. You literally tell your managers that and they are fine

@@Donbros it's unfortunate that so many lessons have to be learned the hard way.

Especially difficult when the software is connected to a hardware device that *can* get stuck, wear out, etc. Very few people write software that accounts for brown-outs, power flickers, or other stuff where the hardware doesn't work to specs.

@@darrennew8211 Excellent points yeah. Kyle mentioned that this particular device didn't have any mechanical fail safes or ways of verifying the software and the hardware were in agreement. I'm sure modern devices have safety switches and sensors in place to confirm the actual state of the machine rather than the state of the software.

People aren't surprised by the fact that video games are buggy but they will believe that the code handling eg. their banking transactions must be rock solid.

As a technical writer of software and hardware, documentation and USING it in QA testing is critical. This concurrent activity finds many opportunities for clarity and correction.

As a high schooler who used to program in grade 9 & 10, diagnosing errors and oversights to a software and fixing them is IMPORTANT for me to keep my grades. Teaches you to be mindful of your code at a very young age

as a bystander that hasnt graduated primary school. AKSJDIAWOESDHUAKSHJCOUAHDELJAHO

@@alphanumeric6582 and most of your idiots in charge don't understand that you can write a program and if even if you let it sit for a while step back and look at it you're going to miss some glaring error that someone else could see and fix easily it's just the nature of the Beast

Documentation and validation? Boeing: never heard about it, all regulator in our pocket, by the way here our new MCAS software.

"Common sense" is what one is forced to rely on when one doesn't know any better. It is what lies beyond rigorous and tested methodology. If engineers are depending on their common sense, they should fucking stop, back away, and tell the client that they are not the engineer for the job or the job is beyond modern engineering. Perhaps that is what an engineer DID do, and why they used some highschool kid or whatever their cover story is.

@@Barnaclebeard You misunderstood their point entirely. The point is "common sense" in a situation like this where an engineer doesn't understand the technology they're dealing with, it's to get help to solve the problem, especially when human lives are at risk. Common sense isn't a synonym for "just try pressing random buttons", in fact, it's quite the opposite lol.

@@Rezu55 I think you're misunderstanding.. Common sense simply means sensibilities that are "common" to everyone (of an average intelligence). To some extent it might mean "flipping random switches" or, more beneficially it may mean having the awareness to back away from the system. The statement "common sense isn't taught in schools any more" is a prejudicial misnomer. It doesn't mean the same to anyone and it really never has been..

@@Rezu55 You don't hire an engineer so that they can practice common sense. Anyone can do that. It's an engineer's job exactly never to use their common sense, because people are stupid. If an engineer is using common sense, people die. That's the end of it. It's not "common sense" to refuse to work on a project with human lives at risk without an adequate history of testing; that's engineering.

"Common Sense" by Thomas Paine, that's a good book. That's how I have mine.

THUMBS UP TO WHAT???? THOSE PEOPLE ARE DEAD!! and the guys who stayed to figure it out did no good!

@@karmendimas5274remove the m in your first name. That’s what you’re acting like

​@@DelGTAGrndrsthat's a good one.

​@@karmendimas5274So... You can't read?

so, you cannot comprehend?@@phoenix9531

At least with game devs you just get a buggy game; it's not going to kill you in one of the worst ways imaginable.

They should be prosecuted for murder. They didn't put a safety on their gun.

@@JoshSweetvale The gun lobby would take offense to that statement and blame the operator.

It’s not like that. Get some life experience outside of video games so you don’t need to keep trying to draw analogies from them

@@ConernicusRex It's exactly like that, I have plenty of life experience outside of video games but that analogy seemed the most appropriate for the situation. It was a joke dude, you don't have to take it so seriously lmao

Also any pc gamer from the last 3 decades can attest to constant crashes and stuff.

People will forget to hit X on one window and think their entire computer is frozen. It's not just error it's following steps. If it's of that importance REWRITE the whole screen don't make your own "corrections" THEY USED EDIT IN THE WORST WAY

If you worked at AECL, probably you know the therac25 programmer. Right?

@@cloviscareca AECL has many facilities across Canada. I was in Manitoba working at a small research facility.

@@randomnotes thanks for your answer. Have you seen a therac 25 in person?

Test-driven design starts with error handling of five right. Tedious but that's the cost of being thorough.

No shit sherlock! that is on every coder/programmers mind but mistakes still happeen...

I think they mean the perspective of an average person, not a software developer. The general public underestimates how buggy most software is, or at least tends to assume that software in critical areas (like healthcare, finances etc.) is cut from a different cloth somehow. (It isn't)

That's like 80% of coding these days

@@kashmirwillwin3124 java and android cof cof

😭😭😭

when the fuck will people see that cars and roads are objectively death traps

they tested it, they just didn't test it with operators who corrected their own input errors in 7 seconds or less.

mRNA is basically software

Early computing was the wild west. Independent review didn't happen.

If they didn't test for such an obvious race condition, it's as if they hadn't tested it at all. People are more careful about website layout than this company was about deadly radiation.

There are reasons we have restrictions and regulations today. Sad to say, but somebody had to figure out the hard way before real change happens for the industry.

Are you kidding? The industry was barely in its newborn stage. There were not yet any standards. No one understood anything about computers back then. There was no training. There was no understanding of what to do if something didn't work. And they did have miniscule documentation, but no one would have read it because they wouldn't have understood it. No one knew anything about computers. Literally nothing.

I wrote code alone for a medical device. It couldn't hurt anyone thankfully. It had bugs if the user did weird stuff I never thought of, and my arrogant boss refused to acknowledge that I would have blind spots about how someone might use it considering I made it, knew how it worked, and intuitively wouldn't use it in ways that didn't make sense while testing.

Remembering your code must be written for the average user, and that users are the stupidest humans to your code is so important. People will put letters in phone number fields or say their birthday is Cactus is you let them

As a game dev student, I have quickly come to realize how fast a user WILL break your software by doing something you may never think of. I am good at breaking games and software but when you make your own thing, you can try 1000 different things to break it but as soon as someone else who isn't making it gets their hands on it, they find something to break and abuse very fast

@@Jkb2002 It's so easy to have happen because you know the kind of responses the system expects and you can't see past the machine to actually witness the operation.

@@KyleLyre13 exactly

Did you mean: ...importance of software DESIGN? Testing finds defects. Correct design prevents defects.

@@olafzijnbuis correct design can still be implemented imperfectly.

@@olafzijnbuis Humans can still mess something up that’s done “correctly.” Same thing that childproof means, you’re kid hasn’t gotten around it YET.

I agree software testing is a good thing to do. However even with software testing you may not reliably discover concurrency issues, problems talking to hardware, etc. For anything that is potentially dangerous, what is more important is to have multiple safety layers, from primitive but reliably hardware interlocks to more sophisticated software interlock system. E.g. at the LHC we have a lot of software that is written by physicists who also do programming. This software can and will sometimes fail. However there are multiple layers of interlock systems behind that will ensure that the beams are immediately dumped in case safe operating conditions are violated, equipment trips, or - as a last resort - particle loss rates reach abnormal levels.

@@olafzijnbuis As a software tester, I can assure you that even properly designed code can contain problems. It is also very challenging to think of all the crazy scenarios a user might come up with while using the software...

So the hardware safeties were masking coding errors. Then when the hardware safeties were removed (cost cutting measure or blind belief that the software was safe due to assumed lack of malfunctions reported?), Therac-25 started killing people.

It is unfortunate that it can only happen in underdeveloped countries (like the United States) that always put companies ahead of their citizens.

@@cosmefulanito5933 I like to say that the US is the wealthiest third-world country

@@blearghuWell it’s 1st world, because that label means we were on one side of the Cold War. Developing country is what we are

@@cosmefulanito5933 The company that produced the Therac-25 is a Canadian State-Funded Publicly-Traded Laboratory. Stop blaming the US for everything. Politically, Neo-Libertarian Beliefs were popular across the western world at the time. State corporatization and free market policies were wildly popular in the west including Europe, North America, China, Japan, and South Korea. The US is only around 20-50 years behind Europe because they're a literally federation twice the size of the European Confederacy.

The sound of children screaming has been removed

Good shit. Amazing.

@@thewhitefalcon8539 I hate that I know that's real, and I hate that I know where it's from. God bless America, where guns have more rights and protections than children.

@@chrismanuel9768 Firearms do not have rights, but we as Americans have a right to keep in bear arms. By that note, unborn babies have even less rights.

@@Ratkill9000 Firearms have rights in America.

I wish they could teach this in our campuses

Pfizer has investigated their own product (Pfizer-BioNTech Covid-19 bivalent booster) by testing it on 8 mice (and 0 humans). The FDA has approved it.

Most of all never let government police them selfs. The DEAMONratparty proves that.

And republicans wonder why federal regulation exists...

Or governments

@@JJ-ub6lv duaaa

Opiates

AECL is still around today and despite it’s website saying “transparency and accountability” they still deny any wrongdoing

I haven't watched the video yet but this sounds like something that would happen in the U.S.

That repeated already earlier in the US - may I bring up Radium girls? Or Asbestos?

@@arinc9 Oh right. Like arrogance, ignorance, and lack of foresight isn't a rife quality in humans all over the world and shit has, is, and will go on in many places around the world that just staggers belief. The radiation therapy machines that seem to be abandoned and then broken into or carted of to a junkyard. The world is polluting itself and destroying ecosystems just fine (not) without the US's involvement. Got have that palm oil so lets deforest Indonesia and turn it into one big palm tree plantation.

A common theme in every nuclear incident is that the suffering wasn't caused by the radioactive material itself, but by ignorant people who made things go wrong.

Don't forget the moronic judge in the Goiana incident siding with the landlord and that if the Fukushima operators wanted to leave the plant on and if they had the dessel generators would not have been needed and the plant would have been fine but they obayed the prime minister who had no idea why hr was making a moronic call also that the prefecture level government actually killed more people with unnecessary evacuations than died of the radiation. Or how failures to communicate lead to mass panic and misinformation about 3 mile island

As the son of a person who holds a Doctorate in Nuke-You-Lahr Physics, I approve this message. My dad has one of the worst cases of "Smoll-Diqk Syndrome" you've ever seen. He was my inspiration for the phrase, "There is no one as stupid as the person who thinks they're smort."

Just think, these are only the stories we’ve heard about. No doubt the Soviets had many unreported incidents, same as the US. Kinda freaky to think about.

And now you have the Russians using the biggest reactor complex in Europe as a Fire Base. "sigh"

I know im just a rando idiot but is everything ok or getting better?

@@j_eezus_christ_bro_chill yeah, I got covid right after I finished treatment so I feel like I've been hit by a bus, but I'm doing OK 👍 😁

we have bug testing today lol

Understandable. My mom went through it over a decade ago and I'm pretty sure even with her being as calm as she was wouldn't have wanted to see something like this. She got a sunburn from her treatment but otherwise was just fine. Also has been cancer free for over a decade now.

So are you okay now?

Medical device software is heavily regulated for this very reason. Very rigorous testing and lots of documentation and approvals. Not to mention comprehensive tech support, user manuals and training.

well they taught you wrong then.

@@robmaelstorm23 warmer, but still not hot. the lesson to be learned here as more to do with all the people involved, not anything to do with the code. The company that shirked it off, the regulators that were lazy. those two were the ones in charge. the bill goes to them, not the software. you could remove all code and programmers from the world and you'd still find that people still make these overconfident or greedy mistake with mechanical devices.

Fun fact: cosmic rays can flip bits arbitrarily

@@Layarion eh, factual, but that's still a very valuable lesson to drill into the head of someone who might otherwise treat something dangerous with flippancy

I had to look up "furl", and it is indeed a word. Vocabulary +1 Meaning: To become rolled up. I had heard of "unfurled" before, so that makes total sense.

@@gopsaysgodwantedyoutoberap7782 who is she

i know. i love it all! I am working on making a fusion reactor in my drawing room. using deuterium and about 40 thousand volts. I will be able to make my favourite type of radiation - neutron radiation. At the minute i can only make x-rays, so I am looking forward to making another type of ionising radiation. I want to 'catch them all', as it were.

Yeah, but if you're looking at death, you'll try anything. Including risk being microwaved by that machine. Hell, you'd stick a hot curling iron up your prison wallet if you think it would help.

@@HollieMoodie Prison wallet is a new one lmfao

Why that is terrible.

@@JacobP81 half melted hand.

Can't tell if that would look badass or sickening.

​@@soupcangaming662 I think cool

@@JacobP81bruh💀

"an overdoes wasn't possible" Yeah they should have explained how those patients suffered from radiation poisoning immediately after the treatment if it "wasn't possible" God that part pisses me off to no end

Companies don't want more money, they want all of the money all of the time. If they can avoid a fine/lawsuit they will, if they can avoid making a new model with intensive testing to ensure it's safe they will. The answer to the question, "How can people be this heartless." Is and so long as capitalism stays the system we use will always be a higher profit margin. If burning your home down made them more money compared to not burning your house down they would do it.

No kidding. It's not like radiation poisoning and sickness is a common thing. Everything comes back to radiation treatment. Honestly the families should have sued and the hospital should have sued the company. The hospital is also at fault for continuing treatment when deadly accidents were clearly continuing to happen.

That's the nature of capitalism for you

@@the_phantom_cat7912 Omega based take.

This and DuPont. Absolutely terrible people

Late Stage Capitalism Fail. Corporate Greed, evil manager lies, the usual stuff. When will people learn to vote for their interests, just like the rich 1% does.

That’s what happens when healthcare is privatized and profited on in a capitalistic system. Has nothing to do with saving peoples lives and everything to do with making money.

​@@fullmetal3233 if anything, it's about only helping people when theyre at their worst. after all, why use cheap methods to prevent sickness when the real money is in trying to treat it?

Well… um… because, uh, if we didn’t, then we’d have slow, overcrowded hospitals, like the UK!!! I know one guy who moved to America from there, he said hospitals were insufferable there!! It’s not like the NHS is the most popular and highly approved government service as voted by UK citizens!

Hobbyists are most brilliant engineers but require proper education. It's also my way of life. Joyful productivity is one side, but learning good habits and discipline is another side. As hobbyist I was chaotic in my early projects, but studying teached me keeping all work in order.

The other issue really though it was code written for a different machine in the same product line. With enough changes to the actual device, the code is impractical to be using without a revision.

@@callak_9974 Yep. I think about same issue, they remove some sensors in new generation and program goes south with errors.

@@adamw.8579 Proper education these days tends to not actually prepare people for their actual work. At the end of the day, at least 90% of what a person learns will be learned on his own, without a dedicated teacher. I have known several computer science majors with degrees from proper universities that don't know how to organize and optimize code. Some of them can't even write code without a helper program leading them along. Every software engineer has several dumpsters full of bad test code that they have produced on their way to writing better and safer code. Idk if this is fundamental to human nature, or if the educational system is just bad in general. But it is a weird trend that I have seen.

@@electroninja8768 I have other habit: plan twice, made once. It's more effective but often not understood by employer. I'm lucky to work on contract with my former client who understand some cartefully planning hours can save many days later. Just he knows my work style.

I think the usual response is "That wasn't supposed to happen."

This might be inappropriate but might i say, "overconfidence is a slow and insidious killer"

Ditto the Titanic

@@davidtitanium22 That or it kills you so fast that you don't even realize that your supposed to keel over!

EXACTLY. “Careful consideration” is not usually all that careful.

yeah this or even the error number AND text saying what it is

If your programmer is a good one, you probably won't get errors, and if you do, they will have an explanation. Poor programers don't add notes to error codes, and you get more of them.

@@jesterprivilege I don't hink it's as simple as good and bad programmers. Usually the higher ups decide if you actually have time to implement error handling. I'm not a professional programmer, but I do some things for my company. My boss would call it waisted time to implement error management or even explanations for people who don't know how to use my little helpers.

I will actually defend this a little bit. Back in the eighties, every single byte of stored memory was precious. It was common practice that error codes would be listed in a book. Adding them into the computer memory might have cost hundreds per computer. A companion book costs... maybe ten dollars. Still the fact this wasn't documented anywhere is an egregious oversight. Literally killed people.

@@TheYear-dm9op Kinda true but it also depends on what is being programmed and for what use. I do believe it should be implemented later on.

came here to see if somebody already said this. Thanks. For some of the math behind this, the maximum value of an unsigned 8-bit integer (colloquially, uint8_t) is 255. The maximum value of any unsigned integer is (2^b - 1) where b is the number of bits. The number of values that an 8-bit integer can hold is 2^8 or 256, but the maximum is 2^8 - 1 or 255 since we count from 0 instead of 1. Some of the "safe software" that we learn in uni is to avoid using magic numbers in code and instead rely on flags, vectors, and enumerated types.

Was going to say the same thing but checked comments first. Exactly right.

Yup.

@Vive le Dominique Fabre no, because a computer cannot actually store a negative value, per se. You would be able to have 256 values aka 0-255 (0 counts) you could have -128 to 127.

It the same with when I see people typing out IP addresses and they put above 255 and act like they are threatening And with most colour scales it’s 255.255.255 as the max number meaning you can only have 16million colours

I really liked this too, helped me get a concept of just how bad these accidents were compared to some of the most insane disasters in human history. It's like the doctor on RU-vid who goes over cases and breaks complicated medical things into understandable bits for the average viewer. I think bring able to teach people complicated things in a simple way is a sign of real intelligence

True. They say the best experts can teach the layperson in a way they can understand.

I thought that too, it was incredibly helpful to understand to have these cases compared to other disasters, it helps people understand the sheer scale of how badly they fucked up

ironically, that could actually be better

Reply bots