bedtime reading

i have put together the following list of some of the papers i found useful, informative or just interesting during the development of the robot project. they come from a variety of sites, and range from the anecdotal to the analytical and mathematical:

• introduction to gambler's ruin - gives the maths behind risk of ruin calculations
  
• risk formulas for proportional betting - this is heavy on the maths but basically contains a lot of very good formulas on bet sizing and risk of ruin
  
• a perspective on quantitative finance - this is by e. o. thorp, and deals more with stock market and hedge fund trading
  
• a new interpretation of information rate - this is the original kelly criterion paper by j. l. kelly jr., published in 1956
  
• the theory of gambling (1, 2, 3, 4) - a book by e. o. thorp with a lot of interesting stories as well as easy to understand maths
  
• the kelly criterion in blackjack, sports betting, and the stock market - the best explanation of the kelly criterion
  
• the first wearable computer - how to win at roulette using a toe-operated computer, another e. o. thorp paper
  
• a guide to casino mathematics - from the university of nevada at las vegas, gaming studies research center!
  

i also started to write a paper myself, on the design and implementation of the robot. i explain how i wrote the software, what problems i found and how i solved them, the maths and reasoning behind the project and so on. i don't know whether or when to release it, since it would certainly make it easier for someone else to duplicate my program, although i believe that anyone capable and motivated enough would not need the information anyway. i'll decide what to do once it's finished.

and, i've now put some more capital into the robot's account and i'm running it 24x7 so i'm hoping that it should show some good results over the next few days...

strategic policy decisions

so, as it turns out, all was not right with my robot at all! i now need to re-seed it with more capital, but having done plenty of analysis of my initial trading (interestingly, it turned over more than GBP 30K of trades on just GBP 250 initial capital) i know what an optimal (and profitable) strategy should be. looking at the graph below, should show that if i ignore one type of trade (a 'lay' bet, that is a bet that something will lose) and had just placed normal 'back' trades, the system would be clearly in profit and growing steadily. i also note that the periods when i was risking a larger than advisable (that is, multiple-kelly, rather than fractional-kelly) percentage of my stake, the fluctuations were larger and profit almost negligible.

this shows profit and loss lines in blue for lays, red for backs and green for the total - the dots underneath are volume (number of bets).

i now need to wait until monday, when i can put more money into the account, however i hope that by then i will have access to a server where i can run the system 24x7 (you know who you are - thanks!) and see how the newest strategy works out over this week.

i've read so many papers on gambling, trading, risk management, capital allocation and probability in the last week, that i don't think i really remember any of them that well. it's been quite hard to translate a lot of them into anything that actually makes sense or is useful in the real world. many start off with unrealistic assumptions, or something equivalent to "first, consider a spherical cow..."! i did, however manage to get some good advice on improving my robot performance (financially speaking) and a lot of them are actually quite interesting in their own right, and i'm going to go over them again - i'll choose a few of the best and post some links here.

the importance of qa!

it's not always plain sailing, running a software robot that trades on internet exchanges - there's a lot of variance/volatility in your minute-by-minute earnings, and sometimes it can make you very nervous. it doesn't matter how much i've proved to myself that mathematically i'm sure to make money in the long run - a 75% drop in your bank balance is quite a shock to any belief in statistics. having said that, i'm not all-in with this game yet - still just using my initial GBP 250.00 stake money, but this is the first time in a few weeks that the level has gone below 'break even', and it was up at GBP 1000.00 at one point.

so, i've been busy changing the logic and money management heuristics today, and i think i'll need to sit down with my fellow cabal member and discuss strategy in depth before putting more of my hard earned dollars at risk! maybe a look at the kelly criterion would be useful?

it's not easy money, you know, or everyone would be doing it...

working - not working ...

well, i'm not working anymore (technically)! i, erm, 'resigned' from my job a couple weeks ago, which was a real shame. the resignation was kindof forced, due to issues that were generally my fault, so i should have really avoided it. anyway, i enjoyed the work, and met a bunch of great people. still, the interesting thing is that this may have actually been a good thing...

i've now started work on a new project that is just as interesting as security work, but should be more profitable, at the very least. i'm reluctant to give away technical details, but in broad strokes i'm running an automated trading system on an internet based exchange. the system determines the correct value for trades and places them automatically. i'm not sure how long it'll take for me to be able to run the software from a remote server and retire to a beach sipping pina colada, since everything is still being tested, but that's the general idea.

the software is written in java, with linkages to some native c code for speed. it's been a fair bit of development, but i've had some help from a friend who's doing a similar thing, and so far we're both in the black. i think the best part has been just seeing the thing work, and knowing that the software i developed is doing it's thing properly and is making money for me. it's different to just finishing an application or a game or whatever, and seeing that running properly - the thrill is not just in getting the software all present and correct and debugged, but in knowing that you've managed in some sort of way to 'beat the system'. and i guess the risk, knowing that if you fucked up - bang go your savings and that beach/drink scenario!

anyway, it's still early days, and i'm going to keep at it until i'm earning a similar amount to what i would as a security consultant (or, at least, as long as it pays my rent and bills, and keeps me in all the bad habits i'm accustomed to). there's also plenty of other electronic markets out there that a software agent could make money from, so i'm also planning on modularising my code, and building a library that can be used on any site that has trading potential

back to 'work' for the moment, then...

london terrorist attack news coverage

well, i'm safe, and importantly, so are all of my friends,,,

however, when watching the rolling news-24 coverage and viewing web news sites, i noticed some things about the disaster reporting that were different to earlier news reports. one thing i discovered was the wikinews collaborative news site. the most interesting was the footage and pictures from the evacuation of an LUL train - they were shot and taken on a camera phone. now, these are images that have never been broadcast before. the tube has been evacuated several times, and people got escorted through the tunnels to safety each time. before now, there was nobody with a camcorder around to document it... now that almost everyone has a camera phone, and video recording mobiles are getting more common, somebody was able to take these pictures

there must be hundreds of other newsworthy events taking place that we only have eyewitness spoken accounts of, which are notoriously unreliable and inaccurate, and these could now be backed up with pictures. also consider official statements and news agency reports about sudden events on the other side of the world - these could be verified by people at the locations being discussed by taking pictures, rather than the usual phone interviedw with a still image of a reporter. viewer's confidence in the truthfulness of the news media would be greatly increased if they knew it was being backed up by evidence from non journalists.

i was able to follow the reporting not only on the BBC site, but also using the flickr photo gallery services. people have been tagging their pictures as londonbombblast or london and bomb so i was able to add an RSS feed of these tagged images to my news aggregator, alongside the official BBC feed. the feed even presents the images in chronological order by timestamp, not in order of upload, to preserve an ordered view of events.
what this means is a lot more news imagery and footage will be available, of things as they happen from an eyewitness perspective, since tools like flickr allow uploading immediately from your mobile. so we will no longer have to wait hours for the camera crews to arrive to verify the official version of events, or be restricted to the one amateur video made by a tourist who happened to be in the right place at the right time. it will also become difficult or impossible to supress or modify images of events since before the police can rip the non-existant film out of the non-existant camera, you have uploaded the picture over the air. i think we are entering an exciting time for news and media, and will see a lot of things that just eye-witnesses experienced before. now, along with the interviews, we will see a slideshow of camera-phone pictures and films as illustration.

the big problem is, of course, managing this information. not as in censorship, but image overload. a flickr RSS feed is fine now, but this will become unmanageable. i think better tagging will allow filtering to be intelligently applied, and when GPS becomes more prevelant, pictures can be sorted by location too, with the attendant possibilities that provides. we will need to have news articles annotated with relevant meta-data to allow automatic linking to picture search engines, and even normal searching. this is just part of the whole 'semantic web' movement, really, where every document is tagged with some meta-data giving it a real-world meaning and context. the semantic tagging allows other programs to provide the links between different content streams in a useful way.

i'm going to the open tech 2005 event on 23 july, so maybe they will have some interesting ideas since BBC backstage (content feed providers) is sponsoring it...

(the other obvious issue is verification of the identity and hence trustworthiness of the photo-taker. this is another problem entirely, however...)

guardian sudoku-ness

i noticed today that the guardian now has its very own sudoku puzzle. since this is puzzle number 24 from saturday, i can surmise that they've had this for less than a month and i've simply not noticed.

the one thing that impresses me is, however, the starting grid for the puzzle. it consists of two squares rotated 45 degrees right in which the numbers 1-9 are placed in order, top to bottom then left to right. one square is at top right, one at bottom left. preserving rotational symmetry (180 degrees about the diagonal) while removing reflectional, two numbers are also placed in the top-left and bottom right squares.

which brings me to another question - is the starting setup of a sudoku puzzle copyright? i guess it probably is, and apologies to the guardioan for the following (although it is fair use, as a quote for review purposes or a citation for educational purposes perhaps?) i also assume that the finishing grid might be copyrighted/able, since it is concievable someone could solve your puzzle then produce another starting position that is derived from your finhing grid... or should i just be pleased with the guardian's hard work on the design of (at the very least) puzzle number twenty four, medium:

...|...|1..                           1  
.5.|..2|.4.                      5   2 4 
6..|.3.|5.7                     6   3 5 7
---+---+---                          6 8 
...|..6|.8.                       1   9  
..1|...|9..                      2 4     
.2.|4..|...                     3 5 7   6
---+---+---                      6 8   2 
3.5|.7.|..6                       9      
.6.|8..|.2.
..9|...|...

now, i've been trundling around the web and found some interesting sudoku related things... firstly, this is an automated web sudoku solver which also gives a list of the steps it took to find the solution. there's a blog which gives the solutions to various newspaper puzzles as well. regarding my query about the total number of sudoku grids possible, this blog thread seems to have come up with an answer - the maths is as hard as i thought it would be, and i'm sure it would have taken me a lot longet to work it out - i was still nowhere near. the number is apparently 6,670,903,752,021,072,936,960 which is equivalent to 9! x 72^2 x 2^7 x 27,704,267,971. see also the wikipedia entry for sudoku for a lot of interesting information...

phone scam hoaxes

i recieved the following email at work today:

Phone Fraud - Warning
We have been advised of a telephone fraud currently in operation - this applies to home and work telephones, landlines and mobiles.

If you receive one of these fraudulent calls, upon answering the telephone you will hear a recorded message congratulating you on winning an all expenses trip to an exotic location. You will then be asked to press 9 to hear further details. If you press 9 you will be connected to a premium rate line that costs approximately ?20 per minute. Even if you disconnect immediately, it will remain connected for a minimum of 5 minutes costing around ?100. The final part of the call involves you being asked to key in your postcode and house number, which has other serious consequences. After a further 2 minutes you will receive a message informing you that you are not one of the lucky winners. The total bill by then will be ?100 (though Trading Standards have confirmed to me that there's some question about the actual costs, but they're still extortionate) .

Since the calls are originating from outside the UK, BT and other telephone companies are left relatively powerless to act. The only safe solution is to HANG UP before the message prompts you to dial 9, even safer HANG UP on any unsolicited 'free offer calls'.

There is another scam operating on mobile phones as well. A missed call comes up. The number is 0709 020 3840, the last four numbers may vary, but certainly the first four will remain the same. If you call this number back you will be charged ?50 per minute. People have complained about their phone bills, once they have realised the cost of the call, but apparently this is completely legal. So beware, do not call back numbers beginning with '0709'.

needless to say, it's all a hoax or urban myth. basically, there are no phone lines that cost more that GBP 1.50 per minute. any that do cost this much will start '09' - these numbers are regulated by ICSTIS. anyway, when called by someone, pressing digits on your phone will never cause any amount to be added to your bill - this is just impossible. as long as someone else calls you, they will be paying for the call. the number '9' has no special functions when pressed during a call.

any phone number beginning with '0709' is a 'find me anywhere' personal number, as defined by OFCOM, and they have been allocated to various companies. admittedly, the pricing structure for these numbers can be somewhat confusing, ranging from local rate to mobile-equivalent rate, and including fixed price per call. however the BT pricelist lists the rate bands. these will be charged by BT at no more than 50p per minute.

a summary of the charges, in pence per minute is shown below:

0709 01         k       37.5p
0709 02         pn2     50.0p
0709 03         pn3     49.0p
0709 04         pn4     26.0p
0709 05         pn6     20.0p
0709 06         j       32.0p
0709 07         pn2     50.0p
0709 08         k       37.5p
0709 09         k       37.5p
07091           k       37.5p
0709 20         k       37.5p
0709 21         k       37.5p
0709 22         k       37.5p
0709 23         k       37.5p
0709 24         k       37.5p
0709 25         k       37.5p
0709 26         k       37.5p
0709 27         k       37.5p
0709 28         k       37.5p
0709 29         k       37.5p
0709 40         pn2     50.0p
0709 41         pn4     26.0p
0709 42         k       37.5p
0709 43         k       37.5p
0709 44         pn9     23.5p
0709 45         j       32.0p
0709 46         pn2     50.0p
0709 47         k       37.5p
0709 48         e        5.0p
0709 49         k       37.5p
0709 65         pn2     50.0p
0709 66         pn2     50.0p
0709 67         pn2     50.0p
0709 68         pn7      4.0p (plus 50.0p fixed cost)
0709 69         pn5     17.0p
07099 0         d       16.7p
07099 2         f       30.0p
07099 4         pn2     50.0p
07099 5         k       37.5p
07099 6         k       37.5p
07099 8         pn2     50.0p

ajax applications

i've been looking at 'ajax' web applications, which are basically things like google maps, google gmail, amazon a9 and other interactive pages that don't rely on server round-tripping to deliver a response to the user. ajax applications provide most of the front-end interaction in a javascript layer running on the browser, which fetches XML and XHTML behind the scenes and uses the DOM to display and manipulate the data. the word 'ajax' is actually an acronym, meaning 'asynchronous javascript and XML' (or '... and XMLHTTPRequest' or '... and XHTML/CSS' according to some people) and is best explained by this essay by adaptive path. also woth a look on the technical side is this blog on XMLHTTPRequest development.

anyway, the company 37 signals produce several pretty useful ajax applications. i particularly like tadalists for web browseable todo lists, and 'backpack' for managing notes and other information, although this costs real money for the most useful features...

here is a list of good ajax resources:

• ajax design patterns wiki
  
• aricles and tutorials about ajax
  
• ajaxian, an ajax blog
  
• openajax, another ajax blog
  

su-doku questions...?

spurred on by occular's post, i decided to try and write my own deterministic sudoku solver. at the moment i'm using java, but i think that prolog or any other AI language would be ideal. anyway, i had a few thoughts while doing this, which i don't know the answers to (yet...)

• how many finished 9x9 sudoku grids are there?
  
• how do you create the starting grids - is it simply backtracking from a finished square that fulfills the constraints?
  
• in connection with the previous, it should be noted that most starting grids i've seen have 180 degree rotational symmetry about a diagonal, sometimes 90 degree about both or reflectional about both, just like crossword grids
  
• is there a way to determine the 'difficulty' of a particular starting grid deterministically, or is it purely subjective?
  

if anyone has any ideas, i'd be interested...