[AmYisraelChai]

I share doomerists concerns that competing AIs may (will?) be used for nefarious purposes and will be highly disruptive to mankind if not an outright existential risk (note: I only use the word existential to describe what I believe are actual existential risks). Here you have a novel system that runs at full speed making them exponentially more productive. This system may soon outpace and outmaneuver humans, and have powers of persuasion that enable them to bypass safety brakes. Add recursive self-improvement, a stratospheric knowledge base that will allow them to synthesize insights, draw correlations and become master prognosticators, an unearthly evolutionary trajectory, ***unknown emergent behaviors,*** and coordination at scale whether acting solo or with other AIs and you have a looming disaster in the making. It could explain Fermi's Paradox. It could also be quite good...in the short term.

The brightest among us don't understand what really goes on in the 'black box' and that black box is expected to morph into a singularity.

There is no room for error yet here we are, plowing at lightning speed in our quest in who can create the most god-like machine. Most folks have no idea what is down the pike.