Separating AI Hype from AI Reality 

😂 Thanks for sharing.

Yeah, this is the other serious issue here: AI productizes the act of plagiarism.

Hang on. Should you not have has some clear discussions about using AI tools on day one? I would think every team would do that.

@@michaelnurse9089 It's definitely going to start being a thing, but it's such an idiotic idea that none of my teams would ever think to bring it up. You have to get approval for everything you use.

Yep, use of AI tools can really elevate a developer when used correctly. But when they are used incorrectly, it causes a massive mess.

​@@michaelnurse9089firstly, as someone else said, it literally never occurred to the rest of us to abuse AI like that. Secondly, he wasn't forthcoming about it so no one one was aware of it until very late in the day. Doesn't help that he works in Asia so little opportunity to pair and become aware of the smell.

Computer Science PhD with 30+ years of SW dev experience and currently a working senior mobile app dev. The experience of the original poster was spot on. AI does not work for chatbots. And AI will not work for serious SW dev for long after everyone alive today is dead.

You are welcome.

You are welcome.

There are two more useful categories. What it can do in the future and what it will never be able to do. These are more pertinent than the right now in such a fast moving field.

You're very welcome!

You are welcome.

Thank you!

You are welcome.

You are welcome.

AI isn't the cause for most of the layoffs. I'm not convinced it has caused any real layoffs. The biggest reason for layoffs is because of the Covid rebound. Companies hired LOTS of developers during Covid. Much more than they needed. Now, budgets are being tightened again and those people are being laid off. However, the other part is that people often look to the big companies (Google, Apple, Microsoft, etc.) and think that they are representative of the industry. They aren't. They do employe a lot of developers, but there are thousands of companies out there that hire developers. For example, I worked at a company in Pennsylvania that employed a couple dozen developers. They've been trying to hire developers almost constantly for years. I know of lots of other companies that are in the same boat. I'm not saying that getting a job is easy, but there are a lot of jobs out there.

You are welcome.

Thanks!

It can be distracting (you are spending half of the time you could in each) and confusing, but it can make getting a job (or the next job) easier as well. It all depends on your current skills (depth and breadth) as well as your goals.

Yeah, it is a great tool in the right context and used with the right level of skill on the operator's part. Not just in asking the right question, but more importantly in being able to evaluate the answer.

The biggest opportunity I see is in how AI lowers the bar for solo developers to do more. This will allow more companies to hire one developer to do custom work for them, where they would not have otherwise been able to even hire one developer. A single developer can now do a wider range of tasks, making them more stand-alone. It used to be that businesses needed a few developers in most cases. That's a really expensive proposition. AI will make that less of an issue, which means more businesses can hire a developer. While hiring a single developer (or a smaller team) might sound like not a big deal, the number of businesses in this category far outweigh the businesses that hire larger teams.

Thanks for sharing!

That was the pitch (along with the lies). The problem is that how do you separate the truth from the lies? Even in their demo, you can see the developer telling Devin how to do things with those tools.

Thanks for sharing!

Getting a good score on a test isn't a great indicator of intelligence. In fact, it really highlights the flaws we have in our educational system. As a developer, do you think a multiple-choice test can identify if you are a good developer? A short answer test? Nope. I used to teach C# at a local college. They really wanted me to have tests to assess the students. I did the bare minimum of testing because it didn't tell me much of value. Sure, it told me if they listened when we talked about some concepts. It could tell me if they understood some theory. But the way I assessed the software development skills of a student was to have them build software. A student could recite the principles of DRY, SOLID, and more to me, but if they couldn't identify when and how to implement them in an application, they didn't actually understand them yet. AI treats the world like digital. Ones and zeros. A CRUD application is the same as every other CRUD application. But we know that's not the case. The world is analog. What works for one customer will not work for another. There are thousands, even millions, of data points that humans assess without even thinking about it. The thought is that if we just get enough of those data points into AI that it can think like us. That's simply not the case.

Thanks for sharing!

And also the prompter - many things that fail on LLM's work once you change the prompt.

This hole take seems pretty misinformed on how the human brain works, as well as (and that's the disappointing part) how AI works.

I think you missed the point of the example I was using. The point was that ChatGPT could logically think through what a nine-sided object would entail and create it. It also could not evaluate its own product to determine that it had given me a failing solution. Even in your example, Gemini isn't actually creating a nine-sided shape. It is using Python code to create the shape. Code that probably already exists thousands of times. That's the deception of AI - you can't just look at the output to determine intelligence. You have to look at how it arrived at that output. It didn't actually create anything new, it just imitated and returned an example based upon what it has been told (through training).

Well, improved file search is something that Microsoft has needed for a long time. That's not really an AI thing. Your entire hard drive has been searchable for forever. Microsoft just didn't do a great job at it before. I am concerned, however, about how much AI is being incorporated into the operating system, as well as every other device. Depending on the checks and balances, it could be a good thing or a bad thing. It should definitely be better than Cortana/Siri/Alexa.

@@IAmTimCorey Tim, file searching is one thing. However this is something completely different. It's recommending items that you didn't search for and displaying them to you when you never asked it to find these files. They just show up. How is this not an AI thing?

It might be a function of the AI to show them to you, but it really is just a file search. The difference is that the OS is trying to anticipate your needs. However, don't think that files are ever "hidden" on your hard drive. Whether the OS shows it to you in anticipation of your search or whether you search for it first, that's the only difference here. Either way, the file is discoverable and readily available.

Definitely not with LLMs (our current "AI"). They just don't work that way. As for AGI (the "artificial life" you are talking about), I'm doubtful that will ever happen but even the companies that are actively working on it say we are a long way off.

Follow-up questions would be good. However, that would take a LOT more work to get right and it would be a LOT more expensive. In the end, I'm not sure if it would provide enough output gains to be worth the price. It is cheaper and easier to train humans to ask better questions of the AI.

If I say "there are too many cars on the road", do you assume I'm excluding trucks, vans, tractor trailers, and more? Or do you understand that the term "car" in that context refers to the broader category? The same thing is true here. Do you think anyone was confused by the term AI? Yes, there are lots of specifics inside of the AI field, but that has been the term that everyone uses for the general field. The actual company you referred to is called "OpenAI". Not OpenLLM or OpenML. In other words, we absolutely should use that term because that is what people are familiar with.

Modern LLMs will not become the foundation of AGI. It is a fundamentally different set of activities. That's not just my opinion, it is the opinion of people actively working on creating AGI and LLMs.

@@IAmTimCorey I s of course and this is exactly what I have said ...so I am not sure if I understood your point, or if you understood mine. Foundation of something and part of something is a huge difference... I can be wrong, but LLM easily becomes a part of AGI because we as humans do the same things in our brain... But it is NOT a foundation of our intelligence, just one of the parts

@@IAmTimCorey agree. great - I got your point. thanks. I guess I meant not the crunching numbers way mimicking the brain but the part where our memory and pattern recognition works etc.. which is still not studied well etc... but having in mind how kids learn stuff just repeating all they see and mimicking ... it is still in a lot of ways how predicting model works to some extend... again - I do not go deep... just basically. other than that of course I agree with you that brain and neuro networks does not have much common besides names )) and some very loose analogy so in a nutshell I love AI in all its forms and I think as many forms we create and then combine together is better (without extremes) - then in the end of day just like brain and body has many many parts working together we can possibly create much better intelligent products when we combine many specialized part under some front-lobe AI orchestration... its just my opinion. Thanks again for your awesome job Tim and even taking time for comments. apricate that. Keep it up man!

Devin itself isn't fake. The demo they showed off to launch it was faked.

@@IAmTimCorey if the demo was fake, do you think microsoft will mention them in the keynotes? i am sure MS saw something real..

No, they won't/didn't mention it. That's not something any company would do. The Devin team already talked about how they faked the demo. That's the problem with the "gold rush" we have right now with AI - companies are rushing to get out the door and faking demos in order to catch the public's eye. The leaders in AI will be paid enormous amounts of money. That leads to cutting corners. This isn't an AI-only problem. The company behind Anthem (the game) did the same thing with their demo. There, the developers actually used the demo footage as a guide for how to build the game. In the case of Devin, it is doing some interesting things. However, it isn't nearly as innovative as they said. It is progress, but it isn't a massive leap forward.

That's great! AI does definitely make you more efficient. That's going to be true for all developers who learn to use this new tool. The trick is to understand the results it gives you. There was a study recently that found that ChatGPT gave the wrong answer to programming questions 52% of the time. The worst part was that 39% of the time, the developer didn't notice and just implemented that bad code. That's the real danger.

@@IAmTimCorey I think there's going to be a huge market soon for developers who can fix all the issues caused by AI.

Well, apps are easy, you wouldn't be saying that if you were programming pacemakers or cockpits. The harder the task the less useful(or more lost/confused) the AI.

@@IAmTimCorey Agree 100%. AI is the new skill.

No, because AI does not have reasoning. It cannot evaluate if something is good or bad. It is already trained on millions of GitHub repos, and it is proving that just seeing lots of code doesn't make you a good developer. For example, a recent study found that it is wrong 52% of the time in programming questions (and 39% of the time, the developer asking the question didn't spot the error).

AI will cause developers to be sloppy, but that will cause problems for them.

​@@IAmTimCorey but what about overall humans not developing intelligently due to reduce in use of actual use of human intelligence ? Evolution will hit hard !

Yeah, it is a bit concerning.

A current search engine takes the prompt and tries to find a page that matches that prompt. An LLM takes the prompt and tries to find various pieces that match that prompt. It then puts them together into a result. That's really all that is happening. The "magic" is it determining which pieces it needs, which parts are relevant even inside of a group of similar solutions, and how to assemble them together into a mostly cohesive whole.

Thanks!

I am glad it was helpful.

We also proved you can brute force RSA

How exactly do you make a speaking database intelligent by increasing the data its been fed? You need to change the way it works so it is an intelligence that relies on the database as information, not vice versa (a database relying on an intelligence to tell it what to do, aka prompting) To make things more clear... you can teach a database what it knows but you cant teach a database how to develope new knowledge, only intelligence can accomplish that and it requires a completely different approach.

@@danilodjokic5303 Different things. RSA brute forcing is practically unviable. Doubling your data quality and compute for a 4x improvement is very doable - that is what OpenAi etc is doing and will be doing going forward - probably for the next couple of decades.

To elaborate my point a bit further... try asking it for causalities and correlations e.g. regarding climate change whether its man made or not (big discussion still) and you will see what I mean - it is repeating whats in his database, it is not explaining anything so its not "intelligent" by any means yet.

@@MrArkaneMage There is no discussion. You can measure it. It's that easy.

I was at a talk given by one of the leaders in the AI industry. The question was asked of them "how do we reduce/eliminate hallucinations?" The answer was "we don't know, and we aren't sure we will ever know."

@@IAmTimCorey AI is making progress, but we are a long way away from what is being promised now. Too much hype, and for some applications of the technology, there are lots of copyright concerns.

I think you are trying to split hairs here. If I can get 10 tasks done before I start using a tool and I can get 15 tasks done after using the tool, doesn't that mean the tool made the job easier? AI helps you get more done. That doesn't diminish the job or its complexity. It just means that you can do more because of this additional tool.

I'm not sure what things you are talking about, but I don't see the comparison. The example I used of ChatGPT not understanding what a nine-sided shape might look like and not being able to evaluate its own work - that is something a 7-year-old could probably get right. The way AI evaluates data is fundamentally different than how humans evaluate data.

There are a few problems with the idea that AI will replace junior developers. First, as you pointed out, that's not long-term sustainable because you need new talent in the industry all of the time. Second, companies never look to hire more expensive people. So, if anyone is in danger, it would be the senior developers since AI can make junior developers more capable. The reality is that each level of developer has their place still. Third, since AI allows people to be more capable, it allows the junior developers to do more than they otherwise could have, therefore they are more likely to be hired by companies. Fourth, we aren't lacking for developer jobs. In fact, being able to do more as a single developer means that more companies that could not afford a team of developers might now hire a single developer. This means more companies offering developer jobs, which would be a huge expansion of the market.

@@IAmTimCorey The market is currently saturated with developers and each job posting gets 100+ applications almost immediately. I can't vouch for each candidates credentials, but speaking with recruiters; companies are looking for Sr. dev's at a lower rate. This is most likely caused by the economy slowing over A.I but from my very recent experience job hunting, employers are looking at Sr's over Jr's. It will be interesting to see what happens over the next few years for sure, my guess would be shrinkage as companies try to do more with less. Jr's will have to be self made or do some form of internship.

The economic ebbs and flows are going to happen. However, the software development field is growing faster than most other career fields based upon tracked data. That means that overall, things will get better. I do want to address the idea that 100+ applicants are applying for every job. First, that's probably not true for on-site only jobs. However, that was true when I opened up a position for a senior developer at my company a few months ago (work from home but in the Dallas area). Yes, I was looking for a senior developer because they were going to be the first developer I hired. So, I received over 300 applications for the position. That sounds like a saturated market. However, I can tell you that over 250 of those applicants were not at a level where it was even possible that they could do the job. People over-estimate their skills and don't have anything to show off to prove what they can do. So that meant that I only really had about 50 people to go through. Of those, a significant number did not actually have the skills or experience of a senior developer. In the end, it came down to about 10 candidates who were at the senior developer level. Then it came down to provable skills and the ability to communicate them. That narrowed it down to just a few. So 300+ sounds like an overwhelming number, but for a remote/hybrid job, it turned out that I only had a handful of viable candidates to choose from.

@@IAmTimCorey Yup, I can only speak from personal experience in the Midwest. Working with several recruiters saying companies are leaning more towards smaller teams filled with Sr devs. I'm sure the market will rebound, and it will be interesting to see what part A.I plays.

😂