For a start, very few humans actually do that. Nobody reasons from first principles. We have heuristics, rules-of-thumb, clichés, all of which we have copied from other people, which is exactly what AIs do.
AIs can copy the methods of logical reasoning without having an understanding of the meaning behind those methods.
It's the genius flashes of insight that humans occasionally get that are difficult for AIs to replicate. If you trained an AI on the
Principia Mathematica it could use those rules to derive all sorts of true equations. But
Gödel's brilliance was in realising that the equations could themselves be subject to arithmetical operations, which meant that the equations could be the subjects of equations, which inevitably led to paradox.
That's the kind of thing that would be difficult for AIs to replicate, in my opinion. The sudden insight from seemingly out of nowhere, the completely new way of looking at something.
Roko's Basilisk doesn't involve any of these flashes of insight. It can be derived purely through the application of simple logic.
DeepGo demonstrated that wasn't the case about 10 years ago - the classic example is "move 37" see:
https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol
Wait wait wait. This is different than where I'd imagined AI is at, at this point. (Which is fair enough, if that is indeed the case, happy to change my perspective if that is really how it is.)
We don't all really have to be Godels and Einsteins, right? Us here, for instance. We're capable of insights, humble ones with a small i at any rate even if not necessarily major breakthroughs in math and physics. So that, it is entirely conceivable that Roko's idea, such as it is, might have come from most any of us here.
In addition, us here, most of us are capable of ...well, a certain degree of critical thinking. We can, most of us, work out why exactly Roko's basilisk falls short, and why we needn't take it seriously.
And I was under the impression that present-day AI is not capable of either. That present-day AI, built as it is basis simply statistical probability of words and numbers, can regurgitate and parody, and do that better and better every day: but it
cannot directly suggest Roko's basilisk in the absence of such having already been proposed somewhere, and also that it
cannot critique it directly (as opposed to regurgitating and collating and summarizing and paraphrasing existing critique already available).
So: Am I wrong in thinking that? Basis your actual understanding of how AI actually works, the innards of it as it were: might AI have actually been able to come up with Roko's basilisk? And might it have actually been able to critique it thoroughly on its own steam?
If I'm actually wrong about this,
and if you're sure that's the case basis your actual knowledge of how AI works at present: well then, okay, I'll revise my views on this, then.
eta:
In which case, assuming the above is indeed what you're saying, assuming you answer that with a Yes: then are we aware of any original idea, or at least a kind-of-sort-of original idea, like this basilisk thing, that AI has come out with, in any context? Or are we aware of any original critique it has ever produced, critique of anything at all, off of its own steam and off of its own critical thinking and not just paraphrased and collated?
