Idle Pursuits

I decided to talk about my latest hobby - horse race analytics. Its something I started thinking about twenty years ago and, after several false starts, I believe I have finally worked out the maths of what I wanted to do and coded up an approach that works. I'm not sure why it took so long!

The basics. Horse racing is an uncertain business. Generally speaking it is never possible to accurately predict the results of a horse race, unless you are a) veeeery lucky or b) own all the horses. The best prediction is usually that the favorite will win. But typically the odds the bookmakers will give you on that happening will not be very good, so if you do it every time, you will, in the long run, loose money. If you back the horse where the bookies are giving the best odds, i.e. they pay you the most for a win, you will also loose money overall, because these horses will win less often. Not never, just less often.

There is, surprisingly, one reliable strategy for making money form horse racing, and that is to pick "value winners" which means that you pick horses where the bookmakers are offering "good value". In other words, they are offering better odds than the quality of the horse would suggest. In yet more other words the horse is more likely to win than they think it is. The "true odds" of the horse are "shorter" than the bookies odds. So, all we have to do is work out the true odds and back horses where the bookies are offering longer odds.

Therein, of course, lies the problem. How to calculate the "true odds". How to calculate odds better than the bookies, whose job it is to do this. They have teams devoted to it; observing races, going to stables, timing, observing, etc. This is what I'm trying to do. It won't be easy - but I think I at least have some maths that can help. I'm planning to use the kind of stuff economists and stock traders know about (at least some of them). To drop in a name, Bayes is the key to this. Bayes is the key to a lot of things.

What I plan to do is develop and refine the method over the coming weeks, publishing what I calculate as the true odds, comparing these to the bookmakers odds and highlighting my betting  recommendation. Over the weeks, we'll see if its working and hopefully refine as we go!

Bitcoin analytics - reprise

I saw this twitter post from Justin Seitz @jms_dot_py the other day:

https://twitter.com/jms_dot_py/status/958741474572750848

Its good to see this push to uncover the underside of the crypto currency ecosystem going forward.  It again mentions the point that the blockchain is a public ledger, which lists all accounts and all transactions that have ever occurred - forever. That gives quite a bit of time for us to pick over it. De-anonomizing accounts is the only barrier to getting full context on those transactions. Of course, there is only so far it can be taken right now, but in future, with better tools, who knows what we will uncover.

I first came across Justin Seitz through his Black Hat Python book - which is packed full of ingenuity. He also does a lot in the OSINT space and developed the very good Hunchley tool for OSINT investigations. Definitely not a black-hat, but he does like to look under stones. I follow his Hunchley daily dark web report - which is an example of that.

MtGox - Time Line

A couple of days ago I dusted off some bitcoin trace data I produced a few months back and decided to look at other ways of processing it. I'm interested in what time correlation could tell us about the bitcoin transaction ecosystem. As a very quick starting point I decided to time-bin the transactions following the MtGox bitcoin show (see my previous post).

I modified my python post-processing routine to count the coin flow and the number of transactions in each of my chosen large time steps (5000 seconds) working from the very first transaction after the show point. Doing this I felt a bit like an astronomer analysing the ancient light from just after the big bang (ridiculous thought). Its pretty cool that this data is frozen forever for us to examine.

Here's some graphs I made by exporting csv and plotting in excel...

Here's the first graph - number of transactions per step. You can see the rate ramp up from near zero in the first step up to hunderds of transactions per step. Note, the x axis is time step number. The steps are 5000 seconds - around an hour and a half (daft choice - I should have used hours shouldn't I :?)

If we look at the value transfer rate (above graph) we see a different story. As time goes on, the actual values transferred fall sharply over time. That's because the transations themselves are getting smaller as the coins get more and more split up. Note the log scale!

So by dividing the volume by the transaction rate we get the mean size - this falls form the initial HUGE transaction steeply down. Note, I'm not considering dilution as I'm not quite sure how to handle it. probably  mean size would continue to fall over time if I looked at the amount of coin that really originated from the original pot as some of the later transactions might be getting quite dilute.

Here is the interesting bit though. I looked again at the first graph and saw what seems to be an outlier... The big spike near the start. So I replotted the very early parts - here's what I got...

There's a huge spike of transactions on around timestep 28. A leap up to nearly 900 transactions and then it drops back again. Clearly something happened here. What I'm thinking is that someone ran a script to disperse the coins widely. This probably makes the trail really hard to follow if you do it manually. Fortunately we have computers so its easy!

Now what I'm wondering if this spike corresponds to the strange whirlpool structure on my Gephi coin trace... I think that whole structure may have been formed very quickly by running a script. I'm now trying to imagine what the script code to do that would have looked like and exactly why it was done in the way it was. Next step is to add a time line to the Gephi trace and step through it. That'll let me see when that structure appears on the graph view. I will keep you posted :) 

The Open Source Personal Meme Filter

Memes to avoid:
Memes that people are out to get us
Memes that sections of society are out to get us
Memes that we (the guys in the white hats) are superior to others
Memes that the people of other countries are inferior, dislike us, are plotting against us
Memes that being unkind to people is fun or cool
Memes that its them or us
Memes that to attack the meme is wrong, that to question a meme is wrong, or reject a meme is wrong
Memes that society has been bad to us and we have a right to take revenge
Memes that deny us our free will - our right to choose our path
Memes that justify the unjustifiable

Fear memes, superiority memes, memes that surreder our free will. memes with their own meme defences. Complexes of mutually supporting memes with no substance (houses of meme cards)

In short - memes that bring us fear and give us licence to do bad things to people

Rules: accept only memes that are free standing, testable, cause nobody any problems, bring good to people, and most of all that you believe in

A further observation: This personal meme filter is itself a meme - hopefully one that passes its own acceptance rules. Its okay to attack this meme - but hopefully it is strong.

An open source meme: This meme is published under an LGPL licence. Please feel free to build on it, but do share your results with us all. Together maybe we can build a better world.

Bitcoin - Digital Fortress?

I just thought I'd float this idea: In his science fiction novel of a few years ago Digital Fortress, Dan Brown describes a supercomputer built by the NSA to crack complex codes like public-private key encryption. We know now from the Snowden revelations that they've been doing a lot of funky stuff - some involving code cracking - some more focussed on back dooring, intercepting and hacking.

But Bitcoin - this technology of unknown origin... It involves an ever increasing amount of computer power computing hashes at an exponentially increasing rate. Vast bespoke computers of unprecedented power are now dedicated to this. Its driven phenomenal advances in bespoke computer power - pushing advances in GPU and bespoke ASIC hardware. I guess the two questions that spring to mind are:
1) Who is really behind bitcoin? Anarchists? Bankers? Con Men? Aliens?... The NSA???
2) Could we have helped (lured by a few riches) build a vast Digital Fortress style global computer to help our government security guys crack the most complex codes. I dunno - is this eveb possible?

The Meme Meme

Thought for the day: The term meme was coined by Richard Dawkins in his book The Selfish Gene. A meme is an idea. Ideas exist within the population (who are their hosts). The point he was making was that strong ideas live and multiply and spread - just like strong (or perhaps more aptly, fit for the current environment) genes.
The idea of a meme is, clearly, in itself a meme. the meme meme (or meme2).
Interestingly common use of the term deviates from the original. It has evolved into something new. The meaning in most quarters is now this. Grotesque image plus big white writing plus bad grammar equals meme. Simples.
But the real meaning is far more interesting. Currently our world is populated by some powerful and dangerous memes. I won't put specific names to them here. There are several - and they generally follow the same pattern. "I believe that my set of ideas is the only correct one - believers in all other sets of ideas are therefore by definition wrong". I disagree with this wholeheartedly. Many sets of ideas can get you through life very nicely and do nobody any harm, and hopefully do some good. There are only a few no-no's in my view, but complete intolerance of a diversity of views, cultures, physical characteristics etc. is definitely high on the list of bads. But why so many bad memes? Compare back against the gene example on which the meme meme is based and you immediately see the answer... Because they thrive in the current environment. That's the thing we've got to understand and do something about... What is it about the current environment that allows these quire damaging memes thrive? The environment is pretty much us, the human hosts, or human society. We need to take a serious look at that.

MtGox and Instawallet AGAIN!?

I really should move on from speculating about MtGox and the murky world surrounding it. But its HARD, with so much still obscure.

In a previous post I showed how Instawallet seems to show up prominently on the transaction trail after the MtGox dog-and-pony-show (by which I mean their "show of coins" back when). Anyway, I just found this thread dating back to when Instawallet nearly folded and got taken over. An interesting piece of history on several fronts. But check out post #19 from Mystery Miner. He seems to be reporting some wierdnes whereby when he tired to use Instawallet to buy ganja on Silk Road, the ganja was actually purchased with apparently stolen coins, while his actual coins got used to purchase a "killer wet job". I have know idea what that is, but that aside it does seem he is reporting some kind of Gox-Instawallet fund confusion or interchangeabilit . although how reliable his evidence is though may be called into question by the fact that he may be judged from his post to be:

a) a self-styled shadowy enigma and international cyber-man-of-mystery
b) a self-proclaimed buyer of illegal drugs and frequenter of the deep web
c) maybe just a bit of a sleaze

... he may have actually been onto something there though!