Tim Urban recently wrote a two-part article about Artificial Intelligence and the dangers involved.
I think some of the comments refuting certain points of the article are quite brilliant and I highly recommend reading them. I read about half the article and browsed the rest. I have read Kurzweil. I agree that AI is a danger, because they will be so powerful, but not because they are better than humans. Human intelligence, if taken to include wisdom, intuition, subjective perception, etc. is far, far, far more and different than just processing info power. And, this is a great point someone made which I firmly believe: human intelligence is inextricably woven together with our biology. You cannot have something think like a human unless it is a human. As I said to a friend, we'll end up building an android that will SEEM to act and think and feel like a human, but it's NOT really doing that. It is only a simulation of what a human does, but is something completely different. An incredibly convincing complex "parrot."
The problem is that some humans, including many of the people who are in the AI field, believe that a human is nothing but a machine with processing power. This is because of a flaw in the thinking of the Western mind that has very much affected our scientific world view: that our biology is actually a weakness, a flaw, "sinful" according to religious thought; that intellect divorced from body (this idea goes all the way back to Plato) is superior to our physicality. THIS is their greatest error. To put it in simple, metaphysical (or even psychological) parlance: the computer has no soul. Why? Because it is not a biological life form, but an artificial construct made by man to imitate a biological life form.
Someday, we may make the mistake of giving machines "human rights" just because some scientist was foolish enough to give the machine a life-like human body and face. What you will see before you is not a being, but something akin to a sociopathic intelligent mannequin that could easily be programmed to smile or cry sad tears while blowing you away with a gun. Some people would rather create a fake mate that does everything you want it to than learn to love a real one. We would be better served to learn to love our fellow humans than to try creating a simulation of one that will do everything we say. Make fantastic machines! By all accounts, do. But don't make fake humans or ones that run human affairs. We will be sorry. The choice IS ours, because that's part of being human. At least it still is for now.
Also, I disagree with some of the comments about future shock (that bringing people from the past becomes more and more shocking to them the closer you get to the present due to ever-increasing speed of technological advances). Despite the scientific/historical propaganda, human cultural evolution does not go in a straight, ever rising line. It goes up and down (and may again if we have a major power failure on the Earth for some reason). We would be just as shocked to go BACKWARD in time to the height of the ancient Egyptian kingdom and see technology and culture that was in ways more advanced than things we have now (we still don't really have the technology and know-how to build one of the great pyramids—until the 1970's, we didn't even have a crane capable of lifting some of the rocks on the site). A clueless percent spending all their time on their little cellphone would be stunned speechless for weeks. Someone from the Middle Ages, which was far less advanced, actually might die of shock.
And, if you brought someone from the 1960s to the present, they would be impressed with some things, but after a few days they would realize, "Oh, things were cooler back in the 60s." Just sayin.'