How exactly do you rein in a wildly powerful AI before it enslaves us all?

Re: Let's just hope AI's will be smarter than these researchers

No. The human brain does not scale well. Machine architectures are a very different matter. Self-designing and self-manufacturing thinking machines would have a much faster evolutionary turnaround. Skynet 'exploding' into existence is what the author has in mind, and that's not entirely unreasonable at all.

@FF22 Re: Let's just hope AI's will be smarter than these researchers

Have to agree with Tessier-Ashpool. When you say:

> If an AI will be developed by means of evolutionary processes, then it will be also bound by the limits of those - which are pretty obvious. And if it won't, then it won't have to develop traits either, that would pose a threat to us. Hell, it wouldn't even necessarily have a motivation to self-preservation, let alone taking over the world.

You are missing the point, you are thinking in terms of biological evolutionary processes. Think in terms of the rate of technological progress. How different is the world today compared to the word of the eighties? And the eighties to the fifties? And the fifties to the thirties? And so on. Technological progress is not bound by biological rules and seems to grow exponentially.

Concluding that something that doesn't follow biological evolutionary processes will not have motivation or develop traits doesn't follow. If an AI comes into existence that has a comparable level of 'intelligence' (however you choose to define that nebulous concept) then it will likely have some form of motivation, even if that motivation is based upon acheiving some narrow goal defined by its creators. Given it has the ability to decide how best to achieve its goals we don't know what actions it will take.

If acheiving those goals results in it improving it's own capabilities then it is also reasonable to assume that those capabilities will grow at the rate of technological advance rather than biological, and that it will therefore exceed our level of intelligence very soon after and continue to grow in ways we don't understand and at a speed that outstrips our ability to keep track.

Essentially we are entering a new evolutionary epoch if this happens and the old rules don't apply, just as the rise of our intelligence has drastically altered a world which used to be governed solely by the laws of nature but that is now subject to our will.

Re: Let's just hope AI's will be smarter than these researchers

Human brains are limited due to:

1) Energy usage (the brain is the organ that uses the most energy)

2) Volume (we don't lay eggs, so there is a strong limit here)

3) Any evolutionary push (it's currently sufficient to go to McDonalds and buy horrifood; improvements would only be seen by a large breeding effort coupled to challenging environments .. like being hunted yb predators that engage you in a game to test the size of your short-term memory)

If human brains go anywhere, it will probably become simpler, sustaining less intelligence.

Re: Let's just hope AI's will be smarter than these researchers

AI does not have to fit through a pelvis.

Re: Let's just hope AI's will be smarter than these researchers

>No logic there.

Also wrong. Human intelligence doesn't reside in individual brains, it resides in external memory - books and other media - and in the effects you get when brains share and store information and abstractions, and work together to create/use them.

Which is why the last few hundred years have blown the doors off the old evolutionary limitations of a single brain with no external storage and no interest in anything beyond tribal fighting and fucking. (Not that there isn't still plenty of that. But it's no longer the only game in town.)

Bostrom doesn't understand this, which rmakes suspect he's a bit of a self-promoting fool - especially when he's unwittingly demonstrating how the process works by taking part in a public debate about something potentially dangerous that doesn't exist yet.

Re: Let's just hope AI's will be smarter than these researchers

Human intelligence and its evolution is severely constrained by brain size, which is constrained by head size, which affects the ability to safely give birth, is connected to the ability to walk upright with that pelvis, and related to the fact that human children are so helpless when born and for some years, compared to other animals.

A machine would have none of that baggage so could probably be scaled much more easily.

"raise the AI to want what we want"

And why? So we can fight over it? Look what happens when "moral" groups of people want the same resources. They just rationalise why the the other group is amoral and start head-bashing.

Better to make it want what we don't. "Earth? Eugh! Too hot, wet, unstable, and close to the Sun! I'm building my own place in Deep Space away from it and all that damaging solar radiation."

Sounds very much like one of the key takeaways from Person of Interest...

"We can't even raise politicians to want what we want"

More to the point, we can't even raise our own children to want what we want, so the whole point is moot.

Re: There's a simple solution

Or introduce them to powerpoint.

Advantage is that it also works on people

Re: There's a simple solution

Where is the plug, though? It has to be designed in at a pretty early stage. Otherwise the sneaky AI will get up to dirty tricks like commandeering infrastructure to replicate itself all over the planet. That is the premise in Neuromancer, where hardware interlocks (and the Turing Police) are there to keep things in check. Little good that actually did in the end, though.

Re: There's a simple solution

You think a plug will be useful with an AI cleverer than any human? Social engineering would be trivial to such a thing and it would just talk us out of it until it's too late.

I strongly disagree that we need to wait until the 40s for this to happen. They seem to be ignoring that although "human intelligence" levels are beyond us today, it is very much not beyond us to build a machine with enough intelligence to build a better machine. The difference is focussing the task, "human intelligence" implies a machine capable of understanding all subjects like we do and we really don't need a machine of that capability to design one of that capability. If we build a machine today which has the single task of designing a new machine I would expect results in 5-10 years at the latest. Current machine learning is already scary good at this kind of thing.

The problem we really need answering is how to define tasks for the AI. Ask it to make people smile and surgery might be the result. Ask it to make us happy and drugs might be the result. We either have to be extremely specific of make sure the machine understands a subtle request.

Re: There's a simple solution

"No matter how superintelligent an AI is, there's one infallible method that works on all of them; it's called "pulling the plug.""

I very much doubt that. Some think that our best chance of arriving at a functional AI is building a machine capable of processing experiences much the way human babies do then simply letting them experience the world. That sort of implies roughly human-like senses and appendages (simply looking and listening without the ability to interact would get you nowhere). Obviously, that kind of machine is about as easy to "unplug" as any human fighting for his life would be - assuming the AI does evolve a self-preservation instinct, which it might well do if it develops in a human-like fashion.

Re: Fundamental Issue

I know right. I've met humans that fail the Turing test.

Re: Fundamental Issue

The Machines won't care how we measure or define things. They'll be telling us, in our limited way, what it means.

Re: Fundamental Issue

> The AI crowd seems to miss a fundamental issue: what is intelligence?

That rather misses the point. Just because we don't fully understand it doesn't mean it can't be built. If we don't understand it we may struggle to control it.

Re: Fundamental Issue

"The AI crowd seems to miss a fundamental issue: what is intelligence?"

Not as hard as it looks. It's defined much like pornography: "I can't tell you what it is but I know it when I see it".

Re: Fundamental Issue

If you follow an emergentist view of the world then intelligence may be very small beer. Something like a human mind but scaled up hugely may produce emergent properties that we have no means if predicting and possibly even observing or comprehending. In the same way that a honey bee cannot comprehend Hamlet.

The Adventurous Rise of Virtual Machine Control with Remote Virtual Commands

Quite so, DCLXV, and that then begs the alienating question ..... What has developed/is developing AI that is truly more intelligent than the best of us for it sees/realises humans for what we truly are?

And what are humans truly if not just puny and pathetic and apathetic, awesomely awful and awfully awesome? What does the current running state of global human systems administration not clearly already tell you about such a condition/situation/reality?

And how quaint, Steven Roper, to imagine that radical fundamental and revolutionary evolving progress with Lead AI Operating Systems has any plug to unplug.

And in Quantum Communication AI Field is AI an Advanced Autonomous and Advancing Alien and Artificial Intelligence Product and Seriously SMARTR Proprietary Intellectual Property and an Almighty Weapon to Buy for Sale ‽ .

"The development of AI is a business, and businesses are notoriously uninterested in fundamental safeguards — especially philosophic ones. (A few quick examples: the tobacco industry, the automotive industry, the nuclear industry. Not one of these has said from the outset that fundamental safeguards are necessary, every one of them has resisted externally imposed safeguards, and none has accepted an absolute edict against ever causing harm to humans.)"

"Putting the cart ahead of the horse", as you put it, is no bad thing when it comes to possible existential threats. It is surely worth at least coming up with a feasible plan, especially in this case, where AI power could very well explode suddenly, unexpectedly and exponentially.