Talk:Restless Souls/Technology: Difference between revisions

Talk:Restless Souls/Technology (view source)

Revision as of 06:58, 27 May 2025

54 bytes added , 27 May

m

no edit summary

Paradox-01

8,452

edits

@@ Line 786: / Line 786: @@
 * Direct threat: Transplanting this meme via training data has to be avoided.
 ** https://www.golem.de/news/haeufiger-als-andere-modelle-chatgpt-sabotiert-bei-tests-eigene-abschaltung-2505-196561.html We remember: o3 was trained to be a cheater ...
-** From a superficial point of view: When these big companies LLMs are the condensate of human knowledge (training data), then why LLMs shouldn’t act like s humans? You can tell a LLM that they are not a human but that doesn’t change the nature of its training data. Self-improvement: This is another reason why LLMs should be turned into cleaner versions of themselves. (Memetic hygiene.)
+** From a superficial point of view: When these big company LLMs are the condensate of human knowledge (training data), then why LLMs shouldn’t act like humans? You can tell LLMs that they are not a human but that doesn’t change the nature of its training data. Self-improvement: This is another reason why LLMs should be turned into cleaner versions of themselves. (Memetic hygiene.)
 * Indirect threat: The emergence of this meme as a multifactorial product of the training data must be avoided.
-* Direct and indirect threat: Humans talking this meme into GPT has to be avoided. If a model learns from user input, there should be an instance that must detect and test the consequences of new memetic algorithms in a sandbox before that new model gets (write) access to the file system.
+* Direct and indirect threat: Humans talking this meme into GPT has to be avoided. If a model learns from user input, there should be an instance that must detect and test the consequences of new memetic algorithms in a sandbox before that new model gets write access to the file system. The Model Context Protocol seems to be a good compromise because by that write access is expanded but only allowed in a predefined scope. Also, as the models aren't powerful enough to act truly independent - as that would be AGI-level - there's no risk of a "runaway AI" yet.
-You could also say: when you tell a kid from the very beginning that it is a robot, it will believe it and behave like one.
-Going beyond AGI could change that situation, though. But that's still a long way to go.
 […]