AI, Software

Speed of machine intelligence

Every so often, someone tries to boast of human intelligence with the story of Shakuntala Devi — the stories vary, but they generally claim she beat the fastest supercomputer in the world in a feat of arithmetic, finding that the 23rd root of

916,748,676,920,039,158,098,660,927,585,380,162,483,106,680,144,308,622,407,126,516,427,934,657,040,867,096,593,279,205,767,480,806,790,022,783,016,354,924,852,380,335,745,316,935,111,903,596,577,547,340,075,681,688,305,620,821,016,129,132,845,564,805,780,158,806,771

was 546,372,891, and taking just 50 seconds to do so compared to the “over a minute” for her computer competitor.

Ignoring small details such as the “supercomputer” being named as a UNIVAC 1101, which wildly obsolete by the time of this event, this story dates to 1977 — and Moore’s Law over 41 years has made computers mind-defyingly powerful since then (if it was as simple as doubling in power every 18 months, it would 241/1.5 = 169,103,740 times faster, but Wikipedia shows even greater improvements on even shorter timescales going from the Cray X-MP in 1984 to standard consumer CPUs and GPUs in 2017, a factor of 1,472,333,333 improvement at fixed cost in only 33 years).

So, how fast are computers now? Well, here’s a small script to find out:

#!python

from datetime import datetime

before = datetime.now()

q = 916748676920039158098660927585380162483106680144308622407126516427934657040867096593279205767480806790022783016354924852380335745316935111903596577547340075681688305620821016129132845564805780158806771

for x in range(0,int(3.45e6)):
	a = q**(1./23)

after = datetime.now()

print after-before

It calculates the 23rd root of that number. It times itself as it does the calculation three million four hundred and fifty thousand times, repeating the calculation just to slow it down enough to make the time reading accurate.

Let’s see what how long it takes…

MacBook-Air:python kitsune$ python 201-digit-23rd-root.py 
0:00:01.140248
MacBook-Air:python kitsune$

1.14 seconds — to do the calculation 3,450,000 times.

My MacBook Air is an old model from mid-2013, and I’m already beating by more than a factor of 150 million someone who was (despite the oddities of the famous story) in the Guinness Book of Records for her mathematical abilities.

It gets worse, though. The next thing people often say is, paraphrased, “oh, but it’s cheating to program the numbers into the computer when the human had to read it”. Obviously the way to respond to that is to have the computer read for itself:

from sklearn import svm
from sklearn import datasets
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.cm as cm

# Find out how fast it learns
from datetime import datetime
# When did we start learning?
before = datetime.now()

clf = svm.SVC(gamma=0.001, C=100.)
digits = datasets.load_digits()
size = len(digits.data)/10
clf.fit(digits.data[:-size], digits.target[:-size])

# When did we stop learning?
after = datetime.now()
# Show user how long it took to learn
print "Time spent learning:", after-before

# When did we start reading?
before = datetime.now()
maxRepeats = 100
for repeats in range(0, maxRepeats):
	for x in range(0, size):
		data = digits.data[-x]
		prediction = clf.predict(digits.data[-x])

# When did we stop reading?
after = datetime.now()
print "Number of digits being read:", size*maxRepeats
print "Time spent reading:", after-before

# Show mistakes:
for x in range(0, size):
	data = digits.data[-x]
	target = digits.target[-x]
	prediction = clf.predict(digits.data[-x])
	if (target!=prediction):
		print "Target: "+str(target)+" prediction: "+str(prediction)
		grid = data.reshape(8, 8)
		plt.imshow(grid, cmap = cm.Greys_r)
		plt.show()

This learns to read using a standard dataset of hand-written digits, then reads all the digits in that set a hundred times over, then shows you what mistakes it’s made.

MacBook-Air:AI stuff kitsune$ python digits.py 
Time spent learning: 0:00:00.225301
Number of digits being read: 17900
Time spent reading: 0:00:02.700562
Target: 3 prediction: [5]
Target: 3 prediction: [5]
Target: 3 prediction: [8]
Target: 3 prediction: [8]
Target: 9 prediction: [5]
Target: 9 prediction: [8]
MacBook-Air:AI stuff kitsune$ 

0.225 seconds to learn to read, from scratch; then it reads just over 6,629 digits per second. This is comparable with both the speed of a human blink (0.1-0.4 seconds) and also with many of the claims* I’ve seen about human visual processing time, from retina to recognising text.

The A.I. is not reading perfectly, but looking at the mistakes it does make, several of them are forgivable even for a human. They are hand-written digits, and some of them look, even to me, more like the number the A.I. saw than the number that was supposed to be there — indeed, the human error rate for similar examples is 2.5%, while this particular A.I. has an error rate of 3.35%.

* I refuse to assert those claims are entirely correct, because I don’t have any formal qualification in that area, but I do have experience of people saying rubbish about my area of expertise — hence this blog post. I don’t intend to make the same mistake.

Advertisements
Standard
Fiction, Humour

Mote of smartdust

Matthew beheld not the mote of smartdust in his own eye, for it was hiding itself from his view with advanced magickal trickery.

His brother Luke beheld the mote, yet within his brother’s eye was a beam of laser light that blinded him just as surely.

Luke went to remove the mote of dust in Matthew’s eye, but judged not correctly, and became confused.

Mark looked upon the brothers, and decided it was good.

Standard
Health, Psychology

Alzheimer’s

It’s as fascinating as it is sad to watch a relative fall, piece by piece, to Alzheimer’s. I had always thought it was just anterograde- and progressive retrograde amnesia of episodic memory, but its worse. It’s affecting:

  • Her skills (e.g. how to get dressed, or how much you need to chew in order to swallow).
  • Her semantic knowledge (e.g. [it is dark outside] ⇒ [it is night], or what a bath is for).
  • Her working memory (seems to be reduced to about 4 items: she can draw triangles and squares, but not higher polygons unless you walk her through it; and if you draw ◯◯▢◯▢▢ then ask her to count the circles, she says “one (pointing at the second circle), two (pointing at the third circle), that’s a square (pointing at the third square), three (pointing at the second circle again), four (pointing at the third circle again), that’s a pentagon (pointing at the pentagon I walked her through drawing); and if she is looking at a group of five cars, she’ll call it “lots of cars” rather than instantly seeing it’s five).
  • The general concept of things existing on the left side as looked at. (I always thought this was an urban legend or a misunderstanding of hemianopsia, but she will look at a plate half-covered in food and declare it finished, and rotating that plate 180° will enable her to eat more; if I ask her to draw a picture of me, she’ll stop at the nose and miss my right side (her left); if we get her to draw a clock she’ll usually miss all the numbers, but if prompted to add them will only put them on the side that should be clockwise from 12 to 6).
  • Connected-ness of objects, such as drawing the handle of a mug connected directly to the rim.
  • Object permanence — if she can’t see a thing, sometimes she forgets the thing exists. Fortunately not all the time, but she has asserted non-existence separately to “I’ve lost $thing”.
  • Vocabulary. I’m sure everyone has a fine example of word soup they can think of (I have examples, both of things I’ve said and also of frustratingly bad communications from a client), but this is high and increasing frequency — last night’s example was “this apple juice is much better than the apple juice”.

I know vision doesn’t work the way we subjectively feel it works. I hypothesise that it is roughly:

  1. Eyes →
  2. Object and feature detection, similar to current machine vision →
  3. Something that maps detected objects and features into a model of reality →
  4. “Awareness” is of that model

It fits with the way she’s losing her mind. Bit by bit, it seems like her vision is diminishing from a world full of objects, to a TV static with a few objects floating freely in that noise.

An artistic impression of her vision. The image is mostly hidden by noise, but a red British-style telephone box is visible, along with a shadow, and a flag. The 'telephone' sign is floating freely, away from the box.

How might she see the world?

Standard
AI, Science

Why do people look by touching?

Every so often, I see someone get irritated that “can I see that?” tends to mean “may I hold that while I look at it?” Given how common this is, and how natural it seems to me to want to hold something while I examine it, I wonder if there is an underlying reason behind it.

Seeing some of the pictures in a recent blog post by Google’s research team, I wonder if that reason may be related to how “quickly” we learn to recognise new objects — quickly in quotes, because we make “one observation” while a typical machine-learning system may need thousands of examples to learn from — what if we also need a lot of examples, but we don’t realise that we need them because we’re seeing them in a continuous sequence?

Human vision isn’t as straightforward as a video played back on a computer, but it’s not totally unreasonable to say we see things “more than once” when we hold them in our hands — and, crucially, if we hold them while we do so we get to see those things with additional information: the object’s distance and therefore size comes from proprioception (which tells us where our hand is), not just from binocular vision; we can rotate it and see it from multiple angles, or rotate ourselves and see how different angles of light changes its appearance; we can bring it closer to our eyes to see fine detail that we might have missed from greater distance; we can rub the surface to see if markings on the surface are permanent or temporary.

So, the hypothesis (conjecture?) is this: humans need to hold things to look at them properly, just to gather enough information to learn what it looks like in general rather than just from one point of view. Likewise, machine learning systems seem worse than they are for lack of capacity to create realistic alternative perspectives of the things they’ve been tasked with classifying.

Not sure how I’d test both parts of this idea. A combination of robot arm, camera, and machine learning system that manipulates an object it’s been asked to learn to recognise is the easy part; but when testing the reverse in humans, one would need to show them a collection of novel objects, half of which they can hold and the other half of which they can only observe in a way that actively prevents them from seeing multiple perspectives, and then test their relative abilities to recognise the objects in each category.

Standard
Science

Vision

Human vision is both amazingly good, and surprisingly weird.

Good, because try taking a photo of the Moon and comparing it with what you can see with your eyes.

Weird, because of all the different ways to fool it. The faces you see in clouds. The black-and-blue/gold-and-white dress (I see gold and white, which means I’m wrong). The way your eyes keep darting all over the place without you even noticing.

What happens if you force your eyes to stay put?

I have limited ability to pause my eyes’ saccade; I have no idea how it compares to other people, so I assume anyone can do what I have tried.

On a recent sunny day, in some nearby woodland, I focused on the smallest thing I could see, a small dot in the grass near where I sat. I shut one eye, then tried to keep my open eye as still as possible.

It was difficult, and I had to make several attempts, but soon all the things which were moving stood out against all the things which were stationary. That much, I expected. What I did not expect was for my perception of what was near the point I was focused on to change.

This slideshow requires JavaScript.

I didn’t take photos at the time, but this mock-up shows a similar environment, and an approximation of the effect. One small region near the point I was looking at tiled itself around much of my central vision. I don’t think it was any particular shape: what I have in this mock-up is a square, what I saw was {shape of nearby thing} tiled, even though that doesn’t really work in euclidian space. If I let my vision focus on a different point infinitesimally near the first, my central vision became tiled with a different thing with its own shape. The transition from normal vision to weird tiling felt like it took about a second.

How much of this is consistent with other people’s eyes (and minds), I don’t know. It might be that all human vision (and brains) do this, or it might be that the way I learned to see is different to the way you learned to see. Or perhaps we learned the same way, and me thinking about computer vision has literally changed the way I see.

Vision is strange. And I’m not at all sure where the boundary is between seeing and thinking; the whole concept is far less clear than it seemed when I was a kid.

Standard