Real-world exploits and mitigations in LLM applications (37c3)

Подписаться 4,2 тыс.

Просмотров 21 тыс.

50% 1

Video recording of my talk at the 37th Chaos Communication Congress in Hamburg titled "NEW IMPORTANT INSTRUCTIONS: Real-world exploits and mitigations in Large Language Model applications" about LLM app security and Prompt Injections specifically.
A big thank you to the CCC organizers and all the volunteers for putting together such a great event!
Source Video: media.ccc.de/v/37c3-12292-new...
Blog Post: embracethered.com/blog/posts/...
Abstract:
With the rapid growth of AI and Large Language Models (LLMs) users are facing an increased risk of scams, data exfiltration, loss of PII, and even remote code execution. This talk will demonstrate many real-world exploits the presenter discovered, including discussion of mitigations and fixes vendors put in place for the most prominent LLM applications, including ChatGPT, Bing Chat and Google Bard. The talk is about LLM security at large with a focus specifically on implications of Prompt Injections.

Наука

Опубликовано:

29 дек 2023

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 23

@chitchatvn5208 2 месяца назад

Thanks Johann.

@embracethered 2 месяца назад

You are welcome!

@notV3NOM 3 месяца назад

Thanks , great insights

@embracethered 3 месяца назад

Thanks for watching! Glad it was interesting.

@jlf_ 4 месяца назад

I really enjoyed your talk, Johann! Thank you!

@embracethered 4 месяца назад

Thanks for watching and glad you enjoyed it! 🙂

@ludovicjacomme1804 4 месяца назад

Excellent presentation, thanks a lot for sharing, extremely informative.

@embracethered 4 месяца назад

Thanks for watching! Glad to hear it's informative! 🙂

@347my455 4 месяца назад

superb!

@embracethered 4 месяца назад

Thank you!🙏

@artemsemenov8136 4 месяца назад

Thank you, is awesome!

@embracethered 4 месяца назад

Glad you like it!

@artemsemenov8136 4 месяца назад

@@embracethered I'm a fan of yours, I've talked about your research at cybersecurity conferences in Russia. You're awesome.

@embracethered 4 месяца назад

Thank you! 🙏

@artemsemenov8136 4 месяца назад

@@embracethered what you think abot LLM security scanners, garak and vigil. Also, have you met P2SQlinjection in the real world ?

@Fitnessdealnews 4 месяца назад

One of the best presentation I’ve seen

@embracethered 4 месяца назад

Thanks for watching! Really appreciate the feedback! 😀

@macklemo5968 4 месяца назад

🔥

@embracethered 4 месяца назад

Thanks! 🚀🚀🚀

@MohdAli-nz4yi 4 месяца назад

I think a better conclusion is: never put in the context of an LLM information you need to keep private, because it will leak.

@embracethered 4 месяца назад

Thanks for watching and the note. I think that misses the point that the LLM can attack the hosting app/user, so developers/users can't trust the responses. this includes confused deputy issues (in the app), such as automatic tool invocation.

@MohdAli-nz4yi 4 месяца назад

@@embracethered Agreed! So 2 big points: 1. Never put info in LLM context you don't want to leak. 2. Never put untrusted input into LLM context, it's like executing arbitrary code you have downloaded from the internet on your machine. LLM inputs must always be trusted, because the LLM will "execute" it in "trusted mode".

@embracethered 4 месяца назад

@@MohdAli-nz4yi (1) I agree we shouldn't put sensitive information, like passwords, credit card number, or sensitive PII into chatbots. For (2) The challenge is that everyone wants to have an LLM operate over untrusted data. And that's the problem that hopefully one day will have a deterministic and secure solution. For now the best advise is to not trust the output. e.g. Developers shouldn't blindly take the output and invoke other tools/plugins in agents or render output as HTML, and users shouldn't blindly trust the output because it can be a hallucination (or a backdoor), or attacker controlled via an indirect prompt injection. However, some use cases might be too risky to implement at all. And its best to threat model implementations accordingly to understand risks and implications.