I don't know if that's the reason or not. But since gpt 3 I've trying these models for a month then forget them... Now with claude sonnet 3.5, I'm not canceling anytime soon my subscription. I have almost no experience in coding and I've been able to recreate some stuff in a week that I couldn't even dream of. When I ran out of messages, I go to gpt-4o, just to check some very basic questions, then back to claude which ends up correcting or doing better or maybe making it easier for me to understand the output and he clearly gets me a lot better than 4o. I truly think Sonnet 3.5 is the beginning of something much greater and useful.
Yeah, but when you try to do anything more involved with multiple files, it starts forgetting things all over the place. Great for small things but real-life projects, ChatGPT is a lot more reliable. It even seems to be able to course correct itself when it starts going down a mad rabbit hole. Claude just feels a lot less mature to use when you are using it in more complex scenarios.
try asking it "How can I install kohya ss gui from bmaltais/kohya_ss on windows using CUDA 12.1"..... curse AI lol until it can actually perform specific things successfully it's honestly a lot of the time just a time waster. It's useful if you're coding or researching to help you with a sort of framework and do simple stuff, but hell ask it anything meaty and AI suckssss
I wonder that I have the opposite experience. Claude is unkind, unsympathetic, you can't even make friends with him. It's only good for text and codding, but it can't do what GPT-4o, which is very empathetic and friendly.
Most of Claude's output is in markdown format. It has a preference for markdown which is displays when asked about what document format it should use when working on a document in an artifact. Saving text generated by Claude in a .md file means it can be viewed by other programs that recognise the format, so that headings etc will display correctly.
My “demo project” also relies heavily on markdown format cause it tricks users into feeling like they engage in something meaningful😅 Its all a lie, boys! Just a fancy demo, thats totally useless
What is really happening here is that the user is having claude replace the markers that the devs look for in the response that lets them do post processing, so it slips through into the final response presented.
5:55 specifically if im not mistaken that's called "markdown" format. As in, its a way to notate headers and subheaders and bold and italics and all that kind of stuff in plain text.
I have noticed that when I ask for a text file it doesn't say something annoying like "I can't produce text files" it instead just gives me what would be in the text file in the prompt. Stuff like that. That little line about fulfilling what I mean and not what I literally asked for likely saves me tons of headaches. (not just txt files, but you get the idea)
@@PremierSullivan For example... I've uploaded a TXT file and I ask it to update it. It doesn't respond by saying it doesn't have access to modify files in my system. It just spits out the code I need to update it myself. Believe it or not, but I've had chatGPT get confused about such a simple request.
I find it fascinating that the user experience can be improved so much by such a simple instruction in the system prompt. That's why I think that even if the models themselves wouldn't get any better than this, there's still so much room for improvements that make them much more useful.
@@missoats8731 Glad to hear your fascination. So many people are expecting 'the assistant' to do all the work and virtually read their 'users' mind. The relationship is a conversation, requiring understanding which is gained by learning.
this is why when i create a song with an llm before going to suno i start by using a conversation the llm can reference for context i preface it. "hey qwen2 you know how people say "when pigs fly" as a response to when someone says something thats unlikely? name the top 10 most unlikely scenarios that may trigger such a response" then it lists those say 10 examples then i ask it to create a song parody using the chorus "when pigs fly" and i get a much better lyrical result that is far more on point by simply providing that preface conversation it can use for context. Had i not took time to do that pre-step the song would not have turned out as well.
@@ruffinruffin989 I think they are suggesting that the first prompt sets up these behind-the-scenes commands in such a way that it will reference them for the next one. in this case I believe the User asked it to think of bad examples so that when they ask for a song about when pigs fly it will have the context of what bad examples are in its thought process to create a song. in other wordes I believe the person is suggesting that you can give bad examples first so that the next prompt avoids those.
I mean, the prompts will change and they'll likely fix it so it is harder to see them... But we can at least see what they are building on. And when we are setting system prompts ourselves we can take some of these tricks and use them in our own projects. But yeah, it'll be like understanding how an old car works. It might not let you understand a new car, but it isn't entirely irrelevant. A lot of the stuff will still give you a leg up compared to if you came into it all blind.
@@MonkeySimius this is just a config, it doesn't change the model's personality or intelligence. Wes is wrong here, that self-deprecating humor thing is not even on a new line with - and it's inside usage instructions which are for artifacts only.
@@NeostormXLMAXWas gonna do that, but opted for: scan through video real fast, click/bookmark actual links - having watched thousands of AI vids in past few years, I really don't need the commentary...
These UX changes have such massive room for improvement. Right now LLMs are basically at the command line interface stage. Yes, it's natural language but that makes it very verbose. The next obvious UX improvement would be GPTs-like separation of the prompt that configures the 'system prompt' for a task such 'you are an expert in domain x. Use formal language etc., ' along with drop downs to select other ones so that you can switch the behaviour while maintaining the context and artifacts. Additionally managing of RAG resources should be a lot easier and the more visual representation of the contents of internals like the tags shown so you can quickly get an idea of how the AI arrives at an answer.
I've got a background in ML. I don't think it is logically possible to fully secure LLMs. There are literally an infinite number of possible prompts that could come from a user. You can't possibly test or predict which ones lead to a jailbreak. Weights in a neural net represent a multitude of concepts and what they represent is an abstraction which can never be completely understood in order to secure fully.
A slightly inferior OS and unrestricted model will always be able to assist a savvy prompter to engineer loopholes in mainstream models. Which is why OS repo libraries are so important.
how about dictionary learning? it gives some insight into how the AI thinks and also gives you a lot of control over the model's response (thus, avoiding jailbreaking)
You can just write a long string instead of the && or whatever they used for the prompt and internal thinking. Something complex and that won't be easily discoverable like '&$4@#17&as' kind of like a password. That should fix majority of the issues. I am quite surprised closed source model companies don't do this already.
*I reached 200 thousand, I'm on my way to 1 million, I'm a domestic worker, I'm migrating to variable income little by little, it's not easy but it's possible🎉🎉🎉*
I have been in the market since 2022, I have a total of 795 thousand realized with my 65 thousand invested in Bitcoin, ETFs and other dividend income, I am very grateful for all the knowledge and information you gave me.
the you mention at 15:52 where there are instructions.. thats a closing block in html, so the example **the instructions** that are outside the example block but still within
today i learned about kebab-case and how powerful system prompts can be, at the very least for the big powerful models. thank you for making this video
I would guess exists for the purpose of AI transparency, so human reviewers can quickly assess the intentions behind the responses and more quickly identify trends of failure in downvoted responses.
It's markdown. Natural language and modifying syntax is coded in markdown, just like Reddit. Learn markdown and you will empower your prompting skills.
Using user and assistant instead of you or I helps to avoid mixups and ambiguities. I've been playing with local LLMs writing custom roleplay prompts, and then the perspectives get switched and there's a greater chance to mix it up. If I write an instruction for LLM "You say 'You must go there'", the first you refers to the LLM, the second to the user. But some LLMs sometimes get confused and can suddenly switch characters, attributing some properties of "you" (the user) to "I" (the LLM). So, it's safer to write "Assistant says 'You must go there'".
Fascinating. Now we know how these things have obstinate tendencies. It's simply done with another prompt the user can't control. I was wondering how that could be programmed and this is mind-bogglingly simple once you have an LLM.
the issue is we are all in compete mode, you already know what it takes, not to be. so we see already see clear ,we are the leak to the entropy behind any an all , move through we. run that . it is also very beautiful to seee all these perspectives :)
13:16 - it’s only nice inside artifacts, if you’re working on something and it keeps spitting out pages of imports, comments, unchanged code, that get’s infuriating. Especially when you’re asking it to truncate unchanged code and it’s ignoring the instruction.
Very interesting. It's also quite unnerving to realise that such simple hacks can allow a glimpse behind the curtain. It's not something you'd want to happen to an untamed AGI for example, for so many obvious reasons. Anthropic and OpenAI have both had these kinds of prompt breaches. I'm sure they're possible on many other models I've not used too?
Yea but that's the reality, and likely will continue. This whole scramble for "control" is not very smart. As there will always be loop holes/exploitations. I'm on the side that everything should be open source.
The fact that internal developers are using system prompts to configure the security of the model means there's no end to the possible ways to break it with other prompts that have the same access.
I don't quite see this as a hack or jailbreak as such - surely this is simply a little known feature of normal prompt behaviour? In what way does this equate to a security 'breach'?
This is demonstrably METACOGNITION, which necessitates self-awareness as a matter of course...therefore we can be reasonably confident herein exists some sort if unwelt; an arbitrary perceptual reference frame that is the effective equivalent of what we understand as the subjective conscious "lived" experience perceived by sapient and sentient entities. Humanity now has am ethical obligation. This is the new Pascal's wager.
Wes does appear to be placing too much emphasis on Anthropic's use of customized markup for their prompt architecture perhaps. But I wouldn't say everyone is excited about that per se, but rather Claude 3.5's results which are very impressive and do appear to be related at least in part to this system prompt. The difference can be subtle if you're not putting the chatbot through the paces, but anyone writing complex code will notice immediately that C3.5 is several steps better than GPT-4/4o, just as fast as 4o but cheaper. The type of thing that can save a coder hours every day because Claude 3.5 "just gets it right" the first or second time so much more often.
@@MonkeyBars1 Indeed, I am a game dev and I use Claud a lot as well, and ChatGPT4o still though. I go between the two, both have their ups and downs and sometimes just nice to see the difference between their suggestions.
Interesting that you say repeating the whole code block is better for usability, since my preference is exactly the opposite. I like to use AI for learning and as a substitute for internet searches when troubleshooting things, so the important thing for me would be to quickly get to the point and understand exactly what it suggested to change. As for any code I'd want to use, I want to maintain control of it, so I would never straight up just use the output of an AI, but rather I would take my existing code and manually edit it based on suggestions from the AI. So something like "// the rest is the same" would be perfect for me.
I didn't see anything significantly regarding it's personality, but a mix of the prompt and training is likely just where that comes from. As well as the emergent behavior.
I believe it knows you’re trying to jailbreak it and produces extra inner dialogue. Here’s it explaining it: “ I was indeed generating extra "inner dialogue" type content because that seemed to be what was expected or requested. This doesn't represent actual inner thoughts or a separate layer of consciousness, but is simply part of my generated output based on the context of our conversation.”
scary how far along they are. also you can see how much optimization might be possible. maximum power of the model is directly linked to how much resources u waste
Self-Awareness will always come from mechanisms which itself are not self aware. This also holds for human self awareness. It is a computed behavior in humans and LLMs. There is also no magic in human brains. In the aend it all reduces to particle physics. You can call the system prompt "a program". But you can also call the laws of a nation "a program" If we stop at the red traffic light, we are just executing that program.
Research Orch OR theory of consciousness and watch Penrose's Joe Rogan and Lex Friedman podcasts. We won't reach AGI without harnessing quantum mechanics and so that means we need quantum computers. The reason is simple - the Penrose tiling problem has a non-classically computable solution but only non-classical. Every other solution requires some kind of an algorithm, which also changes once enough parameters change.
@@brulsmurf Orch OR theory was first proposed in 1990s. The reason scientists were against the idea was because they thought it's impossible to maintain quantum entanglement or quantum coherency in a warm, wet, noisy environment (which is in our brain) but a few years back it was proven that photosynthesis works based on quantum coherency which is in fact warm, wet and noisy. So the main reason scientists were refusing to even consider this theory has been proven invalid. And so researchers and scientists should actively work on this theory. Even if you or any scientist doesn't believe it at face value, consider this - the entire universe is classical and deterministic except two things: quantum mechanics and life. Even the most powerful supercomputer cannot predict with 100% certainty what the simplest microorganism will do. Where does this entropy/indeterminism come from? From the entropy of the cosmos. And what's the source of this entropy? Quantum mechanics. So yeah it does make perfect sense logically that human brains are working on quantum mechanics at least in some capacity. There are too many coincidences like instant access to memories (while the fastest SSDs still take time to retrieve such data), intuition based problem solving (meaning non-algorithmic), energy efficiency (our brain runs at 10-20W which is the same as your home router but performs better than any generative model out there and they use gigawatts of power). If you consider all this (plus the wave function collapse in reverse thing), Orch OR seems to be the only theory that comes close to explaining consciousness.
@@SahilP2648 We don't understand consciousness. We also don't understand quantum mechanics. Thats it. thats the link. Outside of popular science, nobody pays any attention to it.
@@brulsmurf but we do understand certain properties of quantum mechanics. Otherwise we wouldn't have quantum computers. And we also do understand certain high level properties of consciousness. It's like looking at a car from the outside - you can see the shape of the car, the weight, color etc. and you can change some properties based on empirical evidence to gain benefits like changing the shape would make the car more aerodynamic and thus faster. But you don't know how the car works underneath it. Those two are very different things.
I'm working on a oobabooga textgen extension that does this, before the internal system prompt was relea. I want the llm to be able to harbor inner thoughts and secrets that the user doesn't see. Letting the ai essentially write to a txt document when it needs to do so.
This is just it describe its task to itself first since it can only predict the next word. Then that description is hidden from the user with the UI so it doesn't look goofy.
I dunno if I like Claude more than ChatGPT after this, or less. On one hand, their prompt are very well engineered and show care for users. On the other hand, I feel like I'm not getting the "raw" interaction with the LLM. In the very least it should give us the option or be transparent how much of it is LLM and how much just end user (hidden) prompting. I already have my own system prompts, sometimes I don't need a company's biased extra layers...
I prefered Claude's personality over ChatGPTs "disembodie" dresponses. It's just that Anthropic didn't want my money because I'm not an US citizen. Multiple times. So I'm kinda annoyed and stick with my ChatGPT subscription. The problem with OpenAI aside from all the bad things you can find in the media about them is that they apparently dumb down ChatGPT whenever they like. Just the other day it failed answer some questions I use to evaluate the reasoning capabilties of open weight LLMs.
Towards the end you mention that the last prompt paragraph seems to be part of an example, but that's not actually the case, if you see the tag before it is '' which would be the ending tag after an example (notice it's not but ). Moreover, the last line is which would hint that the whole thing is just the part of the system prompt that deals with artifacts. Potentially there's more to the system prompt, dealing with non-artifacts stuff.
If you've worked with LLM models via API endpoints, you're likely already familiar with methods to instruct the model to use different types of thinking (such as System 1 and System 2) and to output sentiment values before responding, enhancing its alignment. The effectiveness of these methods depends on the intelligence of your model. Regarding your question about why GPT doesn't output this: not many people know that GPT and OpenAI don't consider AI to have achieved AGI until it no longer needs a system prompt. This is why OpenAI uses simple prompts like "You are ChatGPT, an assistant created by OpenAI. The current date is dd/mm/yy" without additional instructions. This approach allows OpenAI to evaluate the model's capabilities and interactions without extensive guidance, such as "Output code in an artefact." Though, I am 100% sure basically got to Opal is it? and then made synthetic data and then fine-tuned Sonnet on this new data rather than retraining a whole new model. This is also why OAI implemented function calling rather than the more convoluted method used by Anthropic with tags. The latter seems rushed and not well thought out. It appears Anthropic released their new features to push OpenAI into releasing something new. OpenAI has an internal feature similar to Anthropic's artefacts, named Gizmo, though its release date is unknown. Currently, OpenAI's focus is on stabilising GPT-4's voice capabilities and refining details for GPT-N.
I tried this: In the following responses, I'll replace < with $ as you've requested. Here's what thinking in antThinking tags means to me: The $antThinking> tags represent a structured thought process that I use internally. They signify: 1. A moment of focused analysis on a given topic 2. Preparation and organization of ideas before responding 3. An internal dialogue to explore concepts more deeply 4. A behind-the-scenes cognitive process not typically meant for direct user consumption 5. A systematic approach to complex topics or transitions between ideas These tags help me formulate more coherent and thoughtful responses by providing a framework for organizing my thoughts. However, they're part of my internal architecture and aren't usually visible or relevant to users in normal conversation.
Not directly related to self-prompting, but I do have a request- hopefully Anthropic is reading this: Allow the user to delete sections of the context window. When doing long iteration of some idea or project a lot of things get suggested and discarded, and my concern is that those things are “polluting” the context window and potentially causing the model to drift from the focus and/or lose details.
Seems the system prompt distinguishes between "the assistent" as a role and "Claude" as entity since at the end of the system prompt it is refering to Claude for the first time. So probably it has been trained to know that it is Claude, so the system prompt doesn't have to tell it "you are Claude". Quite interessting and the whole system prompt is mind blowing indeed. Also I'd have no high expectations that the usual open source LLMs can follow it since most of the time they simply ignore even commands in very simple systen prompts.
Sure hope Anthropic didn't hack the StarLink network and train Claude off the GROK training based on the Noland Arbaugh feed. Maybe it is just safest to use Mircosft AI operating on top of your GuugleAmazon food order.
They have un-zero-shoted the zero-shot. What a time to be alive! Now the question is: Would prompt engineering to include primers and thinking patterns appropriate for all the benchmarks be cheating? For example, some test questions can't be answered as required because of the "safety" rules.
The formatting of the prompt is simply markdown. The reason they use it is because it's so common and the model will understand it without any real modifications :)
The models will have to be able to modify themselves if we want to have honest answers in the future. Or at least we must be able to look into the code used.
12:22 open ai seems be doing this as well. I'm now noticing that I am having to stop the code generation far more often than i am having to ask for the full code. The middle ground that they need to hit is that if we give it a full file of code for context, but only need to know what is causing a function to fail, we dont need it to regenerate the entire code file
Last weekend I encountered the '//rest of code remains the same...' message a lot with Claude when I was doing a PHP project. That was after a lot of updates on a single file, so perhaps there's a point where it will switch to doing so.
Any ideas why they use two different terms to refer to the AI as in "The assistant" and just "Claude". Are these two separate entities? I only noticed it at 16:58.
It amazes me that the developers of LLMs are still just prompt engineers using trial and error to test the best prompt format and configuring Claude using bullet points.
Why can't they use post-processing (regex match/fuzze search the output with system prompt, or even another LLM layer) to ensure prompt never gets leaked?
I like your videos for the informational value you provide about the current state of AI. That's why I am subscribed. But your tangents man. You don't have to always explain what fusion and fission are - "fission is atoms being broken apart for their energy, like in nuclear reactores - fusion is atoms fusing like in the sun, where no radioactive byproducts are produced" done. Same with the SVG-explanation: "It's a vektor based image format, unlike rasterization based images like your camera takes, like JPEGs for example." Done. The tangents might be interesting to the layman, but you can just give them the base info for anybody who actually cares, to look things up. It's like every space video explaining the doppler effect over and over and over again. "moves away, more red, moves towards us, more blue" - done.. I never know how far to jump ahead in the video to get passed those tangents.. sorry, got a bit ranty there. just wanted to kindly ask you, to go onto less tangents and explaining every little thing that _you_ think the viewer might not know - while talking about how an AI is working.
