AI, Software

Speed of machine intelligence

Every so often, someone tries to boast of human intelligence with the story of Shakuntala Devi — the stories vary, but they generally claim she beat the fastest supercomputer in the world in a feat of arithmetic, finding that the 23rd root of

916,748,676,920,039,158,098,660,927,585,380,162,483,106,680,144,308,622,407,126,516,427,934,657,040,867,096,593,279,205,767,480,806,790,022,783,016,354,924,852,380,335,745,316,935,111,903,596,577,547,340,075,681,688,305,620,821,016,129,132,845,564,805,780,158,806,771

was 546,372,891, and taking just 50 seconds to do so compared to the “over a minute” for her computer competitor.

Ignoring small details such as the “supercomputer” being named as a UNIVAC 1101, which wildly obsolete by the time of this event, this story dates to 1977 — and Moore’s Law over 41 years has made computers mind-defyingly powerful since then (if it was as simple as doubling in power every 18 months, it would 241/1.5 = 169,103,740 times faster, but Wikipedia shows even greater improvements on even shorter timescales going from the Cray X-MP in 1984 to standard consumer CPUs and GPUs in 2017, a factor of 1,472,333,333 improvement at fixed cost in only 33 years).

So, how fast are computers now? Well, here’s a small script to find out:

#!python

from datetime import datetime

before = datetime.now()

q = 916748676920039158098660927585380162483106680144308622407126516427934657040867096593279205767480806790022783016354924852380335745316935111903596577547340075681688305620821016129132845564805780158806771

for x in range(0,int(3.45e6)):
	a = q**(1./23)

after = datetime.now()

print after-before

It calculates the 23rd root of that number. It times itself as it does the calculation three million four hundred and fifty thousand times, repeating the calculation just to slow it down enough to make the time reading accurate.

Let’s see what how long it takes…

MacBook-Air:python kitsune$ python 201-digit-23rd-root.py 
0:00:01.140248
MacBook-Air:python kitsune$

1.14 seconds — to do the calculation 3,450,000 times.

My MacBook Air is an old model from mid-2013, and I’m already beating by more than a factor of 150 million someone who was (despite the oddities of the famous story) in the Guinness Book of Records for her mathematical abilities.

It gets worse, though. The next thing people often say is, paraphrased, “oh, but it’s cheating to program the numbers into the computer when the human had to read it”. Obviously the way to respond to that is to have the computer read for itself:

from sklearn import svm
from sklearn import datasets
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.cm as cm

# Find out how fast it learns
from datetime import datetime
# When did we start learning?
before = datetime.now()

clf = svm.SVC(gamma=0.001, C=100.)
digits = datasets.load_digits()
size = len(digits.data)/10
clf.fit(digits.data[:-size], digits.target[:-size])

# When did we stop learning?
after = datetime.now()
# Show user how long it took to learn
print "Time spent learning:", after-before

# When did we start reading?
before = datetime.now()
maxRepeats = 100
for repeats in range(0, maxRepeats):
	for x in range(0, size):
		data = digits.data[-x]
		prediction = clf.predict(digits.data[-x])

# When did we stop reading?
after = datetime.now()
print "Number of digits being read:", size*maxRepeats
print "Time spent reading:", after-before

# Show mistakes:
for x in range(0, size):
	data = digits.data[-x]
	target = digits.target[-x]
	prediction = clf.predict(digits.data[-x])
	if (target!=prediction):
		print "Target: "+str(target)+" prediction: "+str(prediction)
		grid = data.reshape(8, 8)
		plt.imshow(grid, cmap = cm.Greys_r)
		plt.show()

This learns to read using a standard dataset of hand-written digits, then reads all the digits in that set a hundred times over, then shows you what mistakes it’s made.

MacBook-Air:AI stuff kitsune$ python digits.py 
Time spent learning: 0:00:00.225301
Number of digits being read: 17900
Time spent reading: 0:00:02.700562
Target: 3 prediction: [5]
Target: 3 prediction: [5]
Target: 3 prediction: [8]
Target: 3 prediction: [8]
Target: 9 prediction: [5]
Target: 9 prediction: [8]
MacBook-Air:AI stuff kitsune$ 

0.225 seconds to learn to read, from scratch; then it reads just over 6,629 digits per second. This is comparable with both the speed of a human blink (0.1-0.4 seconds) and also with many of the claims* I’ve seen about human visual processing time, from retina to recognising text.

The A.I. is not reading perfectly, but looking at the mistakes it does make, several of them are forgivable even for a human. They are hand-written digits, and some of them look, even to me, more like the number the A.I. saw than the number that was supposed to be there — indeed, the human error rate for similar examples is 2.5%, while this particular A.I. has an error rate of 3.35%.

* I refuse to assert those claims are entirely correct, because I don’t have any formal qualification in that area, but I do have experience of people saying rubbish about my area of expertise — hence this blog post. I don’t intend to make the same mistake.

Advertisements
Standard
Professional

Utility function of meetings

A car will go faster if you lower the weight, for example by removing the passengers, luggage, steering wheel, and driver. If you do this, you will have a bad time and not go to space today anywhere nice ever.

Coders often share jokes about useless meetings; certainly, meetings can feel useless — they disrupt flow state, and nothing much seems to happen in them — but they’re not useless.

Meetings are for the business, not for the coder; they are to make sure that the coder is pointing in the right direction and solving important tasks; without it, the programmer may be more productive… but they’ll be producing random things, not money-making things.

Imagine the business has an internally-developed text editor. Left to themselves, a coder might produce something really well-documented with 100% code coverage in automatic tests, but if the business would’ve been fine with something that crashed every 1000 seconds, only did ASCII, and couldn’t open documents more than 2^15 characters long, then that effort was wasted.

The utility function of a meeting is how well it tells the drivers where the engine of production is pointing. The drivers of a company (just like the drivers of a car) may or may not be paying enough attention, may or may not be skilled at navigating the economic environment, may or may not be disregarding the business equivalents of speed limits — but even if the leaders of your company are wildly incompetent (and they’re probably better than any coder like you or I can realise unless we do a business degree), even in the worst case, the meetings can still do their job.

(Given the frailty of human memory, I bet you need someone recording those meetings or you’ll get Chinese Whispers up the corporate chain of command, which would make the meetings useless no matter how well people communicate).

Standard
Software

I’m updating my six-year-old Runestone code. Objective-C has changed, Cocos2d has effectively been replaced with SpriteKit, and my understanding of the language has improved massively. Net result? It’s embarrassing.

Once this thing is running as it should, I may rewrite from scratch just to see how bad a project has to be for rewrites to be worth it.

Aside
AI, Software, Technology

Automated detection of propaganda and cultural bias

The ability of word2vec to detect relationships between words (for example that “man” is to “king” as “woman” is to “queen”) can already be used to detect biases. Indeed, the biases are so easy to find, so blatant, that they are embarrassing.

Can this automated detection of cultural bias be used to detect deliberate bias, such as propaganda? It depends in part on how large the sample set is, and in part on how little data the model needs to become effective.

I suspect that such a tool would work only for long-form propaganda, and for detecting people who start to believe and repeat that propaganda: individual tweets — or even newspaper articles — are likely to be far too short for these tools, but the combined output of all their tweets (or a year of some journalist’s articles) might be sufficient.

If it is at all possible, it would of course be very useful. For a few hours, until the propagandists started using the same tool the way we now all use spell checkers — they’re professionals, after all, who will use the best tools money can buy.

That’s the problem with A.I., as well as the promise: it’s a tool for thinking faster, and it’s a tool which is very evenly distributed throughout society, not just in the hands of those we approve of.

Of course… are we right about who we approve of, or is our hatred of Them just because of propaganda we’ve fallen for ourselves?

(Note: I’ve seen people, call them Bobs, saying “x is propaganda”, but I’ve never been able to convince any of the Bobs that they are just as likely to fall for propaganda as the people they are convinced have fallen for propaganda. If you have any suggestions, please comment).

Standard
Software

Just because you can, doesn’t mean you should

Python lets programmers redefine initializers at runtime. Don’t do that.

Screen Shot 2017-05-19 at 19.07.53

The C preprocessor lets programmers redefine “true”. Don’t do that.

Screen Shot 2017-05-19 at 17.11.35

Swift lets programmers use very foolish variable names. This may be the lesser sin, but still, don’t do that.

Screen Shot 2017-05-19 at 17.07.26

Given python has a reputation for relatively defect-free code, it’s remarkable how few guards it has for enforcing good code — no type safety, no access modifiers, only enforced indentation.

Standard