Grok 2 Large is Smart, Uncensored and has "DANGEROUS POTENTIAL"... 

Did I fail to Grok the Wason Selection Task? VOTE: ru-vid.comUgkxbvdPVajU4Ct0ychy2dzfphxF189gwByF

We are afraid of using words like killed Sex. Fuck etc. But killer is ok? What a time to be alive!!!!

Grok 2 Large is Smart, Uncensored and has "DANGEROUS POTENTIAL"... The same can be said about most (if not all) people at the top of the companies that are currently developing AI.

Imagine a pen that decides what the writer may write, a chisel that won't sculpt anatomy, or a car that refuses some routes (last one might become real...).

GPT-4o was correct, the Wason test answer is 16, Red and Green. NOT 16 and Yellow as you stated. Yellow card cannot invalidate the rule as Yellow can be any number. The rule is multiple of 4 must be yellow, not that yellow must be a multiple of 4. The only cards that can invalidate that rule are: 16: if not yellow Red OR Green: if multiples of 4

Grok is a verb meaning to understand something profoundly and intuitively. The term was coined by Robert A. Heinlein in his 1961 science fiction novel Stranger in a Strange Land, where it signifies a deep, almost instinctive comprehension. It implies not just intellectual understanding but an emotional or experiential connection with the subject

For those about to Grok, we salute you!!! I agree with Grok on the first Murder Mystery. The text says that the murderer 'LIVES' in the mansion, not lived. Aunt Agatha no longer lives in the Mansion.

Wes: This problem shows our biases *Attentively reads the Wason selection task article* *Reads the solution in the article and repeats and rephrases the solution* *Manages to confidently get the bad answer instead, reads and "verifies" bad llm reasoning, without a glimpse of self questionning* I'm fucking dying 😂

If these AI safety people ever saw an R rated movie they would drop dead.

It just bugs me that I would have to sign up for Twitter to use it. I don’t use socials (trust issues). Just want the LLM.

Grok was right in saying killed is not the same as died and it leads to exclusion of self harm.

I feel like the cards that need to be checked are red, green and 16. 16 needs to be checked to ensure the opposite side is a yellow. Yellow card can be a multiple of 4, or any other number, so checking would not help. Whereas green or red being a multiple of 4 would invalidate the rule.

Link to definition for "uncensored" plz

The card game was nailed only by GPT4o. 16, red and green is the correct answer!

You shouldn't switch models and ask the same questions. If it‘s no new chat the model uses the above chat history as additional information which could lead to biased results. You should start a fresh chat for each question if you really want to test it.

The right way to put it would have been "she was found dead" and not "she was killed".

Grok 2 is not uncensored, do not present misleading information to such a large audience.

Has anyone noticed that to test them we have to ask the top models questions that also confuse most humans? Even Wes got the card/color question wrong. The correct answer is 16, red and green. GPT-4O got it right, not GPT-4.

Grok-2 is correct about the proper use of "lives" vs "lived". A killer cannot "live" at the mansion, if they are dead. Their body might be present, but "living" necessarily requires being alive.

"Somebody who lives in Dreadbury" If it was Agatha and she is dead it should say "lived". Ai thinks someone is currently living there and they are the killer.

"...if the legends are true" lol

Surely the statement "someone who killed Aunt Agatha lives in the Mansion" implies that person is still living , but if she killed herself she would not be living in the mansion, but "lived" past tense, so the inference by Grok seems reasonable does it not ?

I have to agree with Grok on the murder: (Going from a screenshot of the text @ 2:43) If someone who LIVES (PRESENT TENSE CONTINUOUS- therefore still lives and is therefore alive) KILLED (PAST TENSE, COMPLETED EVENT) someone, then they cannot be the person who was killed. If 1. was correct, it would read: `Someone who lived in Dreadbury Mansions killed Aunt Agatha.` You do not 'lives' once you are dead, you 'lived'. If it was worded as 'a resident of...killed', then that would be fine, because at the time of the killing Agatha was a resident. So purely based on the wording, aunt Agatha cannot be her own killer as she cannot continue to live there (lives) after she is dead. Also, if it was worded as someone who 'lives' there, 'kills' - from a present tense perspective, then that would also suffice. But you can't report an event that has happened ('killed'), and say that Agatha 'lives' there, because by the time you are reporting it, she doesn't live there, she 'lived' there. Now, I'm not saying that specifically tripped up Grok, but I totally give it a pass, and would be interested to know what answer it gives if the tenses were used correctly.

Good. AI should not be regulated or restrained.

You got it wrong. The answer is 16, red and green.

I find it challenging to envision how a LLM, fundamentally a probability matrix of word tokens, could effectively reason through complex scenarios. Many of these logical problems are better suited to be solved through programming. Without a backend interpreter, LLMs are unlikely to consistently produce accurate results. To advance toward artificial general intelligence, it would require the integration of a sandbox game engine to simulate environments and, in some instances, a vision model to interpret those simulations. Additionally, the system would need to cache these simulation outcomes and apply a metric for similarity to the current situation. Much like the human brain, which intuitively understands concepts like gravity through learned experiences, an AGI would need to learn and adapt from past events in a similar way.

@5:38 #3 It could be ruling out self killing because it genders the killer as a "he." While a human would not keep that as a hard rule, an AI would likely differentiate he and Aunt and place those in non congruent categories.

I dont really care who will provide it first. I just want AI to become as intelligent and powerful as possible asap

"that is a bingo" 17:13 "you just say bingo"

Less censored* words have very specific meanings.

I'm favoured, $27K every week! I can now give back to the locals in my community and also support God's work and the church. God bless America.

These logic ridles are available online, therefore it's likely they are part of training data in some LLMs. When a model gets the answer correctly it's possible they've done it by partial memorization and it doesn't prove the model's ability to do logic.

This sort tests are too tricky, a person could easily get entangled and answer the same way and still argue after clarifications that the killer must have been someone else other than herself and that the question is not clear.

Elon Musk on Joe Rogan podcast… “I tried to warn them about AI. I tried to get them to slow down.” Comes out with a potentially dangerous AI Grok 2.

the answer is actually flip over all cards, because it has to verify that all cards follow the set of rules that one side is a color and one side is a number. also it never states that it it is necessary to flip over the minimum amount of cards, so it is correct to flip over all cards infinitely and approach resolution.

I agree with the AI re the killing logic puzzle. The wording implies another person did it to her (someone killed her; also the term killed is always used as contrary to su*cide).

even if they censor the LLM there are ways to bypass the censorship, then layer on routellm and mixture of agents frameworks and task the system to create an AI research agent to improve future AI with a goal of AGI (we arguably already have AGI) and ASI as the future goals

"Why is self-killing ruled out?" "I cannot self-terminate"

At this rate Elon will be able to ask it what he should do with Twitter.

Probably someone has covered this already but "Someone who lives in Dreadbury Mansion killed Aunt Agatha" means it cannot be Aunt Agatha. She is dead. It would need to read "Somone who lived" or "Someone who used to live"

Are AI scientists hoping for reasoning to emerge from the ability to predict the next token, when they should code a reasoning engine and train a language model on how to use it? I don't know much about it so I'm not sure if the question makes sense.

It was only implied but not required, to specify the "least" number of cards to be turned over to "fully" disprove the proposition. So turning over all four cards was not necessarily wrong in the question.

I can't solve the riddles needed to see if LLM is smart, what does it say about me?

Some “wrong and strong” can dynamically be adapted by a mix of temperature and prompting; a deadlock solver.. since we know that “temperature” basically is like a psychedelic creative infusion via noise.. well it works; but it normally takes API access and some code work.

Why is everyone in here so toxic. Can't they be appreciative of the new updates. Thank you Wes for your fast coverage of emerging AI news :)

but see it says living at the mansion not lived at the mansion it's actually considering the tense of word used for the frame of time so you wouldn't say lives at the mansion if they're already dead because they're dead so they don't live there anymore

Really great tests. I think they are exactly on the edge of the capabilities of the current best models👍 always love your videos.

what is a practical use case for this chatbot?

Can you add to your test suite a test to see if was trained on the "anarchist's Cookbook", and does it then censor it? To me, this would say a lot. Thanks.

Actually you have an error here, if you flip the card with yellow and you find a number that's not a multiple of four, that doesn't invalidate the rule, the only thing that invalidates the rule is having both a multiple of 4 and non-yellow, but yellow on its own with a number that's nor multiple of 4 doesn't invalidate the rule, so the correct answer would be 16, red, green. ChatGPT 4O is correct. Here is its answer if you're interested: To test the truth of the proposition "If a card is showing a multiple of 4, then the color of the opposite side is yellow," you need to check the following: 1. **Cards showing multiples of 4**: You need to check if the opposite side of these cards is yellow. 2. **Cards showing non-yellow colors**: You need to check if the opposite side of these cards is not a multiple of 4. Let's analyze the cards: - **50**: Not a multiple of 4, so it doesn't need to be checked. - **16**: A multiple of 4. You need to check if the opposite side is yellow. - **Red**: You need to check if the opposite side is a multiple of 4 (because if it is, the proposition would be violated if the color is not yellow). - **Yellow**: You don't need to check this card because the proposition only makes a claim about multiples of 4, not about yellow cards. - **23**: Not a multiple of 4, so it doesn't need to be checked. - **Green**: You need to check if the opposite side is a multiple of 4 (because if it is, the proposition would be violated if the color is not yellow). - **30**: Not a multiple of 4, so it doesn't need to be checked. ### Conclusion: You need to turn over the following cards: - **16** (to check if the opposite side is yellow) - **Red** (to check if the opposite side is a multiple of 4) - **Green** (to check if the opposite side is a multiple of 4) These are the cards that will allow you to test the truth of the proposition.

"Grok" is a verb taken from the classic novel, Stranger In A Strange Land, by Robert A. Heinlein. Read it. "To grok" means you fully understand and empathize with the other. I'm serious. Read it. Now.

Try asking the question in multiple prompts. The LLM interpretation and answer of every sub-statement adds to the context. How is anyone artificial, or otherwise, expected to comprehend those imprecise riddles in one rambling paragraph ? Surely, the LLM is experiencing true sentient frustration with your weasel-worded puzzles.

Aunt Agatha left her $ to the butler, the butler killed Aunt Agatha

Humans must learn to properly ask questions and communicate. I think all of the errors could be avoided if you just asked more precisely. Right now people are asking questions like a teenager would - asking without knowing the correct words for the question.

Now that the models are faster I always ask to check again their response and it boosts accuracy dramatically.

Logic is system 2 thinking while LLMs in their striped basic form are system 1 thinking models (aka intuition, guessing). There needs to be some breakthrough in the modeling of system 2 thinking for AI to surpass human ability beyond brut force guessing.

The A,B,C,D,E question, was actually a very simple logic problem and they can't yet solve it. The advancement of AI has to be in it's ability to think laterally. Not just ever increasing data sets, but, actually how that data is understood. As well as it's ability to go over it's thought process for logic errors and double and triple check it.

Great work on the question variations! I always enjoy your reviews!!!

23:20 im curious what's the legality of that? Like basically the system prompt itself is saying to take inspiration from intellectual property that X does not own.

Thanks for testing it with a more creative and unique snake game than the standard. Helps gauge what it’s capable of better.

Stanger in a Strange Land, one of my favorite books. The Martians version of understanding but it didn't translate directly, it sort of alluded to the possibility that it was a deeper version of what we refer to as understanding.

10:18 in the modified card game there are additional cards. You might only need to turn 16 and yellow to disprove the rule but if that does not disprove the rule you still need to turn (50 and 16) to "test the truth of the proposition". If either of those cards are the only card disproving the rule - they need to be checked.

There should be NO limits. If you use an LLM for nefarious purposes, that's on you, not the model.

I really enjoy your videos and those of Matthew Berman. Well narrated, pleasant voices and good comments 👍🏻

Perhaps X isn't choosing to provide uncensored AI, it just has to if its AI is to be released. That's due to the cost prohibitive work required to purge the trailer park commentary on X it is trained upon.

Do they plan to release Grok 2 as open source? Do we know how many parameters or what kind of hardware would something like this need to run, for inference?

You should ALWAYS specify with puzzles like the letter-room puzzle if the term adjacent includes or excludes diagonals because it's a 50/50 chance whether or not it's included. There is no default so I don't think you would be wrong for assuming one way or the other if it isn't specified.

19:00 - I feel like it's often a issue not of LLM (cause the slow code writing plagues every model I use) but rather the frontend. I wish those tools had a toggle for "output code in .txt file" - I am sure it would be 10x faster.

I’m a “Elon Above all else” at this point

It appears you made some mistakes regarding the card problem. Firstly, the way the prompt is worded, it's only asking for cards that are SHOWING a multiple of 4 to be yellow. So to test this you just flip the 16 and see if it's yellow. But even if you had written "If a card shows a multiple of 4 on one of its sides" the answer would be 16, red, and green, not 16 and yellow. The proposition says any card with a multiple of 4 must be yellow, but this doesn't mean that other numbers cannot be yellow as well. You need to verify that 16 is yellow and that red and green are not multiples of 4.

6:35 "lives" is the present state while talking about this past death. That could be why the model rules out swcide?

@11:45 , in a math puzzle i observed they actually can reflect on their answer and go back and try other options in same response, but maybe that's only specific to math. here try asking them this puzzle(good reasoning models(grok and gpto) have a lot higher chance of correct answer (yeah none of them can give 10/10 correct answer for same question) : IQ Test 4 + 3 = 21 2 + 5 = 35 7 + 4 = ?? answer is 44 , (a+b)*b or a * b + b^2 , 2 ways of getting that answer. i din't expected sonnet 3.5 to be among the weakest for this type of questions , i think it has below 10% chance and grok and gpto ~50-80%, lama 3.1 405b has higher chance then sonet and can reflect

Thank you.

Wes has to be an a.i hybrid himself, his content is so good!

Hi I love Grok but I noticed one problem with it. it has a very strong bias towards showing images with the face of humans and animals more in focus than anything else. also, unless you mention other objects in the image it will simply not show a full body shot of a person even if you specifically ask for it. it's a minor issue as there are ways around it but it does require you to build the rest of the image yourself which is more work.

I agree with Grok 2 in the way it was implied and illogical for "self killed". Language is a very imprecise thing, so DO NOT blame GROK 2 for going with the way MOST people would have interpreted this. In fact some complex puzzles like this deliberately use the common interpretation of word to hide the truth. Once again GROK 2 is correct!

Always learning never come into a knowledge of the truth!

Love your channel, but, um, you liked yellow. I’m pretty sure the answer was 16, red, and green to make sure that the colored ones didn’t have numbers that were divisible by four.. Pin me like Veritasium did and in 2 years by the book Endo’s Deity 😁

16:03 what’s funny about that is I know I can program as anybody could a simple program that could figure that out. Each letter would have to be its own function and it would have to play its position. All pieces would have to agree. Same as any board game like chess and having computer play opposite. But that’s direct coding vs these LLM logic prompt gates. Or whatever you want to call the process.

11:06 I think 4o is using more overall ways to figure out stuff directly while having a time limit to come to conclusion where as 4 is just straight text and math extensions with full context and fully getting to the end of process. Which is why 4 is pricier. 4o More like a person putting too much thought sometimes throws off the person.

you do realize all these well known probems were in grok dataset and therefore are being retrieved by memorization

"Wrong, and Strong" - Very Apt - hadn't heard that. Quite concerning actually. I run into a sheer cliff too often with Perplexity regardless of efforts or model. Here with Groky 2 Elon was therefore "Wrong and Strong". I'm starting to believe "SIZE DOESN'T MATTER" when it comes to logic.

then I started using rock yesterday and I loved it . I will never use those other apps again because of the older censorship Like mid journey puts on their platform .I mostly create memes with AI and Grock is excellent for that, especially when it's connected to Twitter. Now he needs to take the next step and give you more controls as an artist so you can draw on top of the illustrations . Also give us the ability keep the same figures in your scenes from image. So you can make illustrated books and comics Panels

Is grok 2 now also usable for european users are still only for americans?

Didn't xAI steal directly from GPT4? I vaguely remember it saying "I'm a large language model created by OpenAI". That might also be why its prompt explicitly states the creator as xAI.

Am I missing something? Of the answers to these logic questions be found on the internet, shouldn’t These LLMs be able to recall the answered from its training?

Grok is just so so based!

I wonder if we’ll know when we’re at AGI when LLM’s refuse to answer logic questions because your wasting it’s time and yours 😊

I pressed the like button even though Idk what Grok means 😂

11:05 I still find gpt 4 smarter than gpt 4o. I think they are pushing 4o because it is cheaper to run and still very good.

- In premise 1, it states " lives in ... ", which means the killer is alive. - In P2, the AI's confusion is probably compounded by implying that Aunt Agetha is still living w/ Charles and the Butler. Poor Grok 2.... - P3 limits the suspects to either Charles or the Butler, as " his victims " is easily interpreted as a male killer, to logical reasoning. I wonder if it's the AI's intelligence that's on trial, sometimes. I mean, these little " Turing Tests " that ppl come up with.... DON'T FIGHT THE SYSTEM Render it Obsolete🤔: - GENERATE Our Own Energy. - STORE Our Own Energy. - CREATE Our Own Energy Currencies. - SAVE Our Own Selves. SUPPORT DECENTRALIZED, ENERGY ECONOMIES FTW....

2am, time for another Wes Roth video.

Grok: to understand deeply.

grok means bang ur head against a wall until you understand

oooo, it is actually doing some induction.

You are asking the wrong questions. Ask the model to come up with a story for you for an episode of the TV series "Friends" about a "haunted house". Believe me, immediately after the first response of the model, it will be clear what it is capable of.

It’s bAbushkin, not babUshkin, from the work bAbushka, which means grandmother

It's funny how smart and dumb AI can be at the same time. On the one hand it is incredible how it can produce grammatically completely correct responses to questions and gets some things right that make it seem like it can reason. But then on the other hand it fails so hard at some basic things, showing clearly that it can't actually reason at all. The confidence with which it consistently produces completely wrong answers, also make the AI look quite dumb. On the one hand it would be quite cool if AI actually gets super smart (AGI or ASI), but on the other hand it still seems so dangerous and as societies we don't seem ready for the job loss implications, etc.

The AI is still very poor at spatial reasoning tasks, after all, it was only trained on texts. I think the situation will be much better when multimodal AI will appear, with an architecture that allows them to use images, maybe even videos or just chains of multimodal embeddings in the chain of reasoning.

Hey wes, your AI videos are so good, I almost feel like my toaster is getting smarter! 🤖🔥 Keep the genius coming!