Of Mages and Grimoires

When I first got started with Clojure, I didn't know (and it was a while before I was told) about the clojure.repl toolkit which offers Clojure documentation access from within an nREPL instance. Coming from the Python community I assumed that Clojure, like Python, has excellent API documentation with examples that a total n00b like myself could leverage to bootstrap my way into simple competence.

While Clojure does indeed have web documentation hosted on GitHub's Pages service, they are eclipsed in Google PageRank score if not in quality as well by a community built site owned by Zachary Kim: ClojureDocs. This wouldn't be a real problem at all, were it not for the fact that ClojureDocs was last updated when Clojure 1.3 was released. In 2011.

While Clojure's core documentation and core toolkits haven't changed much since the 1.3 removal of clojure.contrib.*, I have recently felt frustrated that newer features of Clojure such as the as->, and cond->> which I find very useful in my day to day Clojure hacking not only were impossible to search for on Google (being made of characters Google tends to ignore) but also didn't have ClojureDocs pages due to being additions since 1.3. Long story short: I finally got hacked off enough to yak shave my own alternative.

**drumroll**

Grimoire

Mission

Grimoire seeks to do what ClojureDocs did, being provide community examples of Clojure's core functions along with their source code and official documentation. However with Grimoire I hope to go farther than ClojureDocs did in a number of ways.

I would like to explore how I find and choose the functions I need, and try to optimize accessing Grimoire accordingly so that I can find the right spanner as quickly as possible. Part of this effort is the recent introduction of a modified Clojure Cheat Sheet (thanks to Andy Fingerhut & contributors) as Grimoire's primary index.

Something else I would like to explore with Grimoire is automated analysis and link generation between examples. In ClojureDocs, (and I admit Grimoire as it stands) examples were static text analyzed for syntax before display to users. As part of my work on Oxcart, I had an idea for how to build a line information table from symbols to binding locations and I'd like to explore providing examples with inline links to the documentation of other functions used.

Finally I'd like to go beyond Clojure's documentation. ClojureDocs and Grimoire at present only present the official documentation of Clojure functions. However some of Clojure's behavior such as the type hierarchy is not always obvious.

< amalloy> ~colls

< clojurebot> colls is http://www.brainonfire.net/files/seqs-and-colls/main.html

Frankly the choice of the word Grimoire is a nod in this direction... a joke as it were on Invoker's fluff and the Archmagus like mastery which I see routinely exhibited on Freenode's Clojure channel of the many ins and outs of the language while I struggle with basic stuff like "Why don't we have clojure.core/atom?" and "why is a Vector not seq? when it is Sequable and routinely used as such?". "Why don't we have clojure.core/sequable?"?

Clojure's core documentation doesn't feature type signatures, even types against Clojure's data interfaces. I personally find that I think to a large degree in terms of the types and structure of what I'm manipulating I find this burdensome. Many docstrings are simply wanting if even present. I think these are usability defects and would like to explore augmenting the "official" Clojure documentation. Andy Fingerhut's thalia is another effort in this direction and one which I hope to explore integrating into Grimoire as non-invasively as possible.

Status

Much of what I have talked about here is work that needs to be done. The first version of Grimoire that I announced originally on the Clojure mailing list was a trivial hierarchical directory structure aimed at users who sorta kinda knew what they were looking for and where to find it because that's all I personally need out of a documentation tool for the most part after two years of non-stop Clojure. Since then I've been delighted to welcome and incorporate criticism and changes from those who have found Grimoire similarly of day to day use, however I think it's important to note that Grimoire is fundamentally user land tooling as is Leiningen and as is Thalia.

As such, I don't expect that Grimoire will ever have any official truck with Rich's sanctioned documentation. This and the hope that we may one day get better official docs mean that I don't really foresee migrating Grimoire off of my personal hosting to a more "clojure-ey" domain. Doing so would lend Grimoire an undue level of "official-ness", forget the fact that I'm now rather attached to the name and that my browser goes to the right page with four keystrokes.

Future

As far as I am concerned, Grimoire is in a good place. It's under my control, I have shall we say a side project as I'm chugging along on "day job" for GSoC and it seems judging by Google Analytics that some 300 other Clojureists have at least bothered to stop by and some 70 have found Grimoire useful enough to come back for more all in the three days that I've had analytics on. There is still basic UI work to be done, which isn't surprising because I claim no skill or taste as a graphic designer, and there's a lot of stuff I'd like to do in terms of improving upon the existing documentation.

Frankly I think it's pretty crazy that I put about 20 hours of effort into something, threw it up on the internet and suddenly for the first time in my life I have honest to god users. From 38 countries. I should try this "front end" stuff more often.

So here's hoping that Grimoire continues to grow, that other people find it useful, and that I manage to accomplish at least some of the above. While working on Oxcart. And summer classes. Yeah.....

^D

Oxcart and Clojure

Well, it's a month into Google Summer of Code, and I still haven't actually written anything about my project, better known as Oxcart beyond what little per function documentation I have written for Oxcart and the interview I did with Eric N.. So it's time to fix that.

Motivation

Clojure isn't "fast", it's simply "fast enough". Rich, while really smart guy with awesome ideas isn't a compilers research team and didn't design Clojure with a laundry list of tricks that he wanted to be able to play in the compiler in mind. Instead in the beginning, Rich designed a language that he wanted to use to do work, built a naive compiler for it, confirmed that JVM JITs could run the resulting code sufficiently fast, and got on with actually building things.

The consequent of Rich's priorities is that Clojure code is in fact fast enough to do just about whatever you want, but it could be faster. Clojure is often criticized for its slow startup time and its large memory footprint. Most of this footprint is not so much a consequent of fundamental limitations of Clojure as a language (some of it is but that's for another time) as it is a consequent of how the existing Clojure compiler runtime pair operate together.

So Clojure wasn't designed to have a sophisticated compiler, it doesn't have such a compiler, and for some applications Clojure is slow compared to other equivalent languages as a result of not having these things. So for GSoC I proposed to build a prototype compiler which would attempt to build Clojure binaries tuned for performance and I got accepted.

Validating complaints

Okay, so I've made grand claims about the performance of Clojure, that it could be faster and soforth. What exactly do I find so distasteful in the language implementation?

Vars are the first and primary whipping boy. Vars, defined over here, are data structures which Clojure uses to represent bindings between symbols and values. These bindings, even when static at compile time, are interred at runtime in a thread shared global bindings table, and then thread local bindings tables contain "local" bindings which take precedence over global bindings. This is why (clojure.core/alter-var-root!) and the other var manipulation functions in the Clojure standard library have the -! postfix used to annotate transactional memory mutation, because only a transaction can modify the root bindings of a thread shared and thread safe Var.

Now Vars are awesome because they are thread shared. This means that if you drop a REPL in a live program you can start re-defining Vars willy nilly and your program will "magically" update. Why does this work? Because Clojure programs never hold a reference to a "function", instead they hold a thread synchronized var which names a function and get the latest function named by the var every time they have to call that function.

This is great because it enables the REPL driven development and debugging pattern upon which many people rely, however for the overwhelming majority of applications the final production form of a program will never redefine a Var. This means that consequently the nontrivial overhead of performing thread synchronization, fetching the function named by a var, checking that the fn is in fact clojure.lang.IFn and then calling the function is wasted overhead that the reference Clojure implementation incurs every single function call. The worst part about this is that Var has a volatile root which poisons the JVM's HotSpot JIT by providing a break point at function boundaries which the JIT can't inline or analyze across.

IFn is another messy part of Clojure programs. The JVM does not have semantics for representing a pointer to a "function". The result is that Clojure doesn't really have "functions" when compiled to JVM bytecode, instead Clojure has classes providing .invoke() methods and instances of those classes, or more often Vars naming IFns may be passed around as values. This isn't a bad thing for the most part, except that IFns use Vars to implement their .invoke() methods and vars are slow.

The real reason that this can be an issue is why do we need an IFn with multiple invoke methods? Because in Clojure functions can have multiple arities, and the single instance of an IFn could be invoked multiple ways where a JVM Method cannot be.

The real issue with IFns is that they exist at all. Every single class referenced by a JVM program must be read from disk, verified, loaded and compiled. This means that there is a load time cost to each individual class in a program. Clojure exacerbates this cost by not only generating a lot of small classes to implement IFns. When a namespace is required or otherwise loaded by a compiled Clojure program, the Clojure runtime loads foo__init.class, which creates instances of every IFn used to back a Var in the namespace foo and installs those definitions in the global var name table. Note that this loading is single threaded, so all synchronization at load time is wasteful. Also note that if there are top level forms like (println "I got loaded!") in a Clojure program those are evaluated when the namespace containing the form is loaded.

The sales pitch

So what's the short version here? Clojure has Vars in order to enable a dynamic rebinding model of programming which deployed applications do not typically need. Because applications do not tend to use dynamic binding for application code, we can discard Vars and directly use the IFn classes to which the Vars refer. This could be a significant win just because it removes that volatile root on Vars that poisons the JIT.

This opens up more opportunities for playing tricks in the compiler, because we don't really need IFn objects for the most part. Remember that IFn objects are only needed because methods aren't first class on the JVM and to support dynamic redefinitions we need a first class value for Vars to point to. If all definitions are static, then we don't need Vars, so we can find the fully qualified class and method that a given function invocation points to, freeing a Clojure compiler to do static method invocation. This should be a performance win as it allows the JVM JIT to escape type checking of the object on which the method is invoked and it allows the JIT to inline in the targeted method among other tricks.

If we can throw away IFns by implementing functions as say static methods on a namespace "class" rather than having seperate classes for each function, then we cut down on program size in terms of classes which should somewhat reduce the memory footprint of Clojure programs on disk and in memory in addition to reducing load time.

Oxcart

So what is Oxcart? Oxcart is a compiler for a subset of Clojure which seeks to implement exactly the performance hat tricks specified above and a few more. For the most part these are simple analysis operations with well defined limitations. In fact, most of what's required to use Oxcart to compile Clojure code is already built and working in that Oxcart can currently rewrite Clojure programs to aid and execute the above transformations.

Oxcart is also a huge exercise in the infrastructure around the clojure.tools.analyzer contrib library, as Oxcart is the first full up compiler to use tools.analyzer, tools.analyzer.jvm and tools.emitter.jvm as more than the direct compilation pipeline which tools.emitter.jvm implements. This means that Oxcart has interesting representational issues in how passes and analysis data are handled and shared between passes, let alone how the data structure describing the entire program is built and the performance limitations of various possible representations thereof.

So what's Oxcart good for? right now: nothing. Oxcart doesn't have an AOT file emitter yet, and relies on tools.emitter.jvm for eval and as such is no faster for evaling code in a live JVM than Clojure is. At present I'm working on building an AOT emitter which will enable me to start doing code generation and profiling Oxcart against Clojure. I hope to post an initial emitter and a trivial benchmark comparing a pair of mutually recursive math functions between Clojure and Oxcart.

Before you go

I've know I've said this entire time that Oxcart is a Clojure compiler. That's a misnomer. Oxcart doesn't compile Clojure and never will. Clojure has stuff like eval, eval-string, resolve, load-string, load and all the bindings stuff that allow Clojure programmers to reach around the compiler's back and change bindings and definitions at runtime. These structures are not and never will be supported by Oxcart because supporting them would require disabling optimizations. Oxcart also doesn't support non-def forms at the top level. Oxcart programs are considered to be a set of defs and an entry point. Oxcart also assumes that definitions are single. Redefining a var is entirely unsupported, abet not yet a source of warnings.

Some of these differences are sufficiently extreme that I'm honestly on the fence about whether Oxcart is really Clojure or some yet undefined "Oxlang" more in the style of Shen, but for now I'll stick to building a prototype :D

^D

Dogecoin block pricing experiment

As I've explained repeatedly before, the block reward is the mechanism which we as a community use to buy the hashing power which keeps our network relatively independant and secure as an honest processor of users transactions. As we cannot pay miners in fiat somehow, we pay them in Doge in the form of block rewards and transaction fees. This means that we as a community are putting a price on the hashing power used to secure our network. I thought it would be interesting to try and figure out what that number is.

The reason it's interesting is the Dogecoin block reward schedule. To encourage mining and manage the value of the coin, Doge operates on a diminishing rewards block schedule that encourages mining and buying into Doge with hashing power by rewarding early miners more than late miners. However this works only to a point. Mining Doge has to be ROI positive or nobody would bother with it. Mining Doge also has to be ROI competitive with other altcoins or nobody would bother with it. This first requirement I address in this post, and the second requirement I'll address some other time.

Experiment

This is current (ish) pricing and power consumption information for a variety of different mining hardware. The important number here is the final column, KH/w. As the point of this exercise is to put a Doge price on every block, the power consumption of every KH pointed at mining Doge counts.

 Vendor      Miner              KH/s       Watt   Cost           KH/w  
-----------+------------------+---------+-------+-----------+---------
 KnCMiner    Mini Titan         150,000     400   $5,495.00    375.00  
 KnCMiner    Titan              300,000     800   $9,995.00    375.00  
 Gridseed    ASIC Blade Miner   5,200        70   $1,049.95     74.29  
 Gridseed    ASIC 5-Chip        350           7   $79.95        50.00  
 DualMiner   ASIC USB 2         70            2   $81.99        46.67  
 GAWMiner    War Machine        54,000    1,280   $5,899.95     42.19  
 GAWMiner    Black Widow        13,000      320   $1,799.95     40.63  
 GAWMiner    Fury               1,000        30   $159.95       33.33  
 DualMiner   ASIC USB 1         70            3   $98.00        28.00  
 Radeon      R9 290X            850         295   $579.99        2.88  
 NVIDIA      GTX 770            220         230   $329.99        0.96  
 NVIDIA      GTX 560 Ti         150         170   $120.00        0.88  
-----------+------------------+---------+-------+-----------+---------
 Average                                                      89.1525  

At present Doge is near 1K Doge = 0.435 USD, or measured in doge/usd 2298.85.

Measuring the price per block

power costs 0.1220 USD/KWhr at US national average prices, so lets run some numbers.

 1 (ns doge
 2   (:require [meajure]
 3             [clojure.algo.generic.arithmetic
 4              :refer [+ - / *]]))
 5 
 6 ;; constants and definitions
 7 ;;------------------------------------------------------------------------------
 8 
 9 (def kw-per-w
10   #meajure/unit [1/1000 :kwatt 1 :watt -1]) ;; definition
11 
12 (def hr-per-block
13   #meajure/unit [1/60 :hour 1 :block -1]) ;; definition
14 
15 
16 ;; simulation variables
17 ;;------------------------------------------------------------------------------
18 
19 (def kh-per-watt
20   #meajure/unit [375
21                  :khash :watt -1 :second -1]) ;; variable
22 
23 (def usd-per-watt-block
24   (* #meajure/unit [0.122 :usd :kwatt -1 :hour -1] ;; variable
25      kw-per-w 
26      hr-per-block))
27 
28 (def doge-per-usd
29   #meajure/unit [2298.85 :doge 1 :usd -1]) ;; variable
30 
31 
32 ;; computation
33 ;;------------------------------------------------------------------------------
34 
35 (def price-per-watt-block
36   (/ doge-per-usd kh-per-watt))
37 
38 (print "DOGE per watt-block |"
39        price-per-watt-block)
40 
41 (def price-per-kh
42   (* usd-per-watt-block 
43      price-per-watt-block))
44 
45 (print "DOGE per KH         |"
46        price-per-kh)
47 
48 ;; break even block reward
49 (print "DOGE for hashrate   |"
50   (* price-per-kh
51      #meajure/unit [50 :mega :khash :second -1]))
DOGE per watt-block | #meajure/unit [25.785592103418296,
                                     :second, :watt, :khash -1, :usd -1, :doge]

DOGE per KH         | #meajure/unit [5.243070394361721E-5,
                                     :doge, :khash -1, :second, :block -1]

DOGE for hashrate   | #meajure/unit [2621.5351971808605,
                                     :doge, :block -1]

Analysis

 Miner              KH/s       Watt      KH/w   Min. block reward  
------------------+---------+-------+---------+-------------------
 Mini Titan         150,000     400    375.00       623.243777777  
 Titan              300,000     800    375.00       623.243777777  
 ASIC Blade Miner   5,200        70     74.29      3158.329954954  
 ASIC 5-Chip        350           7     50.00      4674.328333333  
 ASIC USB 2         70            2     46.67      5007.851224910  
 War Machine        54,000    1,280     42.19      5539.616417790  
 Black Widow        13,000      320     40.63      5752.311510378  
 Fury               1,000        30     33.33      7012.193719371  
 ASIC USB 1         70            3     28.00      8347.014880952  
 R9 290X            850         295      2.88     81151.533564810  
 GTX 770            220         230      0.96    243454.600694440  
 GTX 560 Ti         150         170      0.88    265586.837121212  
------------------+---------+-------+---------+-------------------
 Average                              89.1525           28935.106  

In order to recieve any return on invested compute power, a miner must find blocks. If the difficulty is too high, then it isn't feasable to find blocks and mining is ROI negative. This is why Wafflepool and other multipools switch coins: they mine for as long as it's ROI positive to do so and once the difficulty adjusts rendering it no longer ROI positive to do so they bail out and head for greener pastures.

So, Dogecoin has a minimum block reward of 10,000 Doge, but we won't see that until January. At present we're at a 125,000 Doge block reward. Looking at this table, that indicates that Nvidia GPUs are now absolutely ROI negative. I won't be mining any more for this reason.

You don't just get coins for mining tho, you get coins for finding blocks. Assuming that the Scrypt proof of work function is inviolate, what fraction of the hashrate do each of these miners need to represent at the current block reward to break even?

 Miner                KH/s   Min. block reward   Hashrt. Frac.   Hashrt. (KH/s)  
------------------+--------+-------------------+---------------+----------------
 Mini Titan         150000       623.243777777    4.9859502e-3       30084.5360  
 Titan              300000       623.243777777    4.9859502e-3       60169.0730  
 ASIC Blade Miner     5200      3158.329954954     0.025266640      205804.9700  
 ASIC 5-Chip           350      4674.328333333     0.037394627        9359.6334  
 ASIC USB 2             70      5007.851224910     0.040062810        1747.2564  
 War Machine         54000      5539.616417790     0.044316931     1218495.9000  
 Black Widow         13000      5752.311510378     0.046018492      282495.1300  
 Fury                 1000      7012.193719371     0.056097550       17826.0900  
 ASIC USB 1             70      8347.014880952     0.066776119        1048.2790  
 R9 290X               850     81151.533564810      0.64921227        1309.2790  
 GTX 770               220    243454.600694440       1.9476368         112.9574  
 GTX 560 Ti            150    265586.837121212       2.1246947          70.5983  
------------------+--------+-------------------+---------------+----------------
 Average                             28935.106                                   

And what's our current hashrate? 48GH/s, or 48000000 KH/s.

Conclusion

I think the writing's on the wall here. If everyone on the Dogecoin network was running the War Machine miner, our hashrate is a factor of 48 times higher than would be break even. What does this mean for our future? As block rewards fall, either the price of Doge will rise due to adoption as a unit of trade which will drive down the minimum block reward numbers, decreasing the hashrate fractions and increasing the global break even network hashrate.

What do I think this means? Well after fooling around with this math and the price of Doge variable the computed maximum network hashrate goes over 45GH only when the price of Doge increases to about 1523 Doge/USD, or at current market prices about 130 Satoshi BTC. If the price of Doge doesn't rise back to the 130 mark, I expect that we will wee our hashrate slowly but steadily fall until it reaches a point where miners perceive that they are breaking even, being very likely under 1GH/s according to these numbers.

If we want Dogecoin to grow into something more than a tip currency I think that we missed the boat when we turned down merged mining with LTC. We have to secure our hashrate for the long haul that it'll take us to build real market valuation through acceptance as a unit of trade. Merged mining with LTC would have achieved that goal. So what do I think the outlook is? I think we're gonna run out of fuel in earth orbit rather than get to the moon.

Dogecoin part II

This post originally appeared on reddit.com/r/dogecoin, and was added to my official tech blog for posterity. or something.


Hey shibes, /u/arrdem back again with more science and musing on the future of our beloved coin. last time, I talked in generalities about the impact of ASICs on the price of DOGE and their impact on the dogeconomy in general. Today I'd like to go into some more depth on ASICs, proof of work, proof of stake and the future of DOGE as we look ahead and think about long term valuation. I got some amazing feedback and insightful comments, which I thought warranted a follow up article. So here we go!

From last time

Last time when I wrote, I mentioned the dogecoin block schedule and pointed out that the rate at which we are issuing DOGE is such that most of the second and third generations ASICs which are being manufactured targeting Scrypt coins will never see "high" returns per block simply due to the impending halvening(s).

On valuation

An open question for cryptocurrencies is where do we derive valuation from? What justifies Bitcoin's standing $500 prices? The Bitcoin crowd looks to Austrian economics and the fact that Bitcoin is asymptotically finite resource as the sources of their valuation.

Now I'll confess that I'm a skeptic of both Bitcoin and Dogecoin as they stand today. As a computer scientist, Bitcoin is fascinating because the blockchain and proof of work which Satoshi introduced is a viable solution to the Two Generals Problem. This article gives a good summary of the significance of this discovery. So what does this mean? It means that the Bitcoin protocol which underlies Bitcoin, Dogecoin and all the others represents a fundamental advance in the design of distributed systems. That this advance is used to implement a distributed, verifiable "value" (coin) store or cryptocurrency is just the first of many problems which I expect to see revolutionized by cryptocurrency derived software. Namecoin is a fascinating example of the wider value of the blockchain technology.

So the blockchain itself as a technology is interesting. What does this actually mean for cryptocurrencies? It means that cryptocurrencies are potentially the ultimate value refuge. If we assume that there is no heartbleed scale flaw in Bitcoin derived networks which allows for private keys to be stolen and we assume that coin holding entities are capable of securely storing their private keys it is impossible for computational power less than 51% of the coin network to steal coins. However at the magic 51% tipping point a malicious majority can start writing fake transactions into the blockchain and generally ruin everything.

Another reason to hope for the value of the coins is that they provide more secure transaction capabilities than credit cards do. As we were reminded with the Target hack, in order to process one credit card transactions, companies who accept credit cards need enough information to process an arbitrary number of arbitrary sized trasactions. The cryptocurrencies do much better at this, as it is impossible for a payment recipient to "charge" repeatedly. Thanks to multiple signature transactions we've even solved the escrow problem, although escrow providing payment processors for coin based purchases still need to mature.

The last significant source of value I see is the low transaction costs. I happen to be DLLAzkmxUypWnF6TvdmCVwaDrro1isqiWh, and knowing that address anyone with the dogecoin client and spare doge can throw some my way at a marginal cost per transaction of $0.00 no matter how large the transaction compared to the 5% and 10% fees chared by other online payment processors such as Square and the existing credit card companies. This means that as we see with DOGE the community can at near zero marginal cost donate to causes anywhere in the world, further increasing the potential impact and reach of previously centralized crowdfunding efforts.

In the shadow of the moon

Now there's a dark underside to the blockchain techology: the proof of work function which Satoshi presented. The blockchain only functions because any entity who wishes to intern records in the blockchain must perform a significant and fundimentally worthless computation in order to prove their honesty. Bitcoin uses the sha256 hash function, we use the Scrypt hash function. Some coins, such as Primecoin have taken an interesting approach to the fundamental worthlessness of the sha256sum and Scrypt computations by trying to use scientific computing problems as proofs of work.

Primecoin is in an interesting place, because verifying primality of a number or number sequence is comparatively easy. Other proposals especially for protein folding based coins, are ultimately doomed because the difficulty of verifying a solution to some folding problem is equivalent to solving it in the first place. This means that verifying blocks and the transactions encoded therein is exceedingly slow and hard, which decreases the micropayment utility of the coin.

However, mining is ultimately a network mechanism for securely processing transactions. It has a cost and no value. The per block reward mechanism first used in Bitcoin is a mechanism for purchasing the mining power required to securely process transactions by inflating the coin and expecting that miners will speculate against the future value of the coin to justify their mining costs. Note that this is fundimentally circular. The initial miners are highly rewarded for supporting the network at its most vulneralbe, with miners who join only after the coin has achieved some modicum of stability being less strongly rewarded.

This fundamental "get in early" reward for mining is why we see altcoins or shitcoins pop up and die right off either having been maliciously 51%'d as may have been the case with Arouracoin. What we seem to be seeing with the price of DOGE is that the BTC/MH/Hr return of mining DOGE isn't high enough to attract new miners. Using data from http://liteshack.com/ We're sitting at about ~30% of the script hashing power not pointed at Litecoin, and ~19% of the total Scrypt hashing power. This means that, as /u/listlesscraig pointed out on my original post, we could very easily as a coin be crushed by a malicious 51% attack should someone on the Litecoin or Wafflepool side of things decide that they really really had it in for us.

So what does this mean? It means that in order to be secure as a coin, our hashrate needs to increase significantly. We either need to pay more for mining, or we need to decrease our dependance on mining (more on that later).

With the block 200001 halvening about 48hrs out, we'll see the per block reward drop to 125000Ɖ. At current market prices (115 satoshi as of this writing) that pegs our per block reward at 0.14375BTC/block, or $71.88/block. So at our current network hashrate of a 62.449GH average, which means we're effectively paying a 0.0000000023BTC/Kh/min price for our hashing power. In comparison, mining Litecoin pays out at 0.00000000304BTC/Kh/min by my math. As our return is lower than LTCs and new hyped alts (shitcoins) are even higher, I think it's clear that we are unlikely to see Wafflepool or other significant sources of hashing power come back without a price jump into the 200s at least.

Now I can sit here and run numbers, but clearly DOGE while featuring an awesome community is not at present a high ROI coin which means we are not accumulating the hashing power needed to secure ourselves against potential machinations of present whale miners ignoring the potential threat of future ASICs.

Now, some people including /u/GoodShibe have been known to call for community mining effort. However he's the good shibe and I'm the math shibe. On average, the current generation of graphics cards can do about ~360KH/s, so if we assume that every subscriber on this subreddit has a "real" graphics card, that gives us a combined hashrate of 28,395,000KH/s, which is 44.75% of the current network hashrate. Assuming that error due to shibes rocking hundred GPU rigs and shibes who CPU mine average out, I think this number makes it clear that as a community we can't raise the hashing power that we "need" to armor ourselves against the potential machinations of whales and pools.

Adjusting launch window

As we can't raise the raw hashing power that's required to maintain our beloved coin and our market valuation continues to fall, it seems to me that there are two plans of action. The first is to try to raise awareness of Dogecoin through publicity stunts like Doge4water and Dogecar. I hope that these both prove to have been positive forces in the long run. Ultimately however, Dogecoin is only as useful as Bitcoin or Litecoin is if we can only tip each other, donate and hoard. Payment processor adoption and getting businesses to start accepting cryptocurrencies are the real way to secure and increase our long term valuation.

In the short term we need to secure ourselves as well so we can get to the long term. Clearly as a community we don't have the kind of hashing power we need to do it ourselves with the "classical" proof of work structure, so I suggest that we look elsewhere, especially to proof of stake.

Proof of stake is an alternative transaction verification scheme under which "miners" mine not on the basis of how much computational power they have, but on the basis of how many coins they control. This means that the mining operation can be much less computationally expensive, and that a 51% attack is impossible except from an entity with a significant stake in the coin who would stand to loose much in the fallout from DOGE's destruction.

By adopting proof of stake, we would make it much more difficult for a wild whale to crush us whether out of malice or by accident. However most importantly we increase our own perceived security and stability which clears the way for more adoption down the road.

Afterward

I look forwards to seeing what you think of this piece. Frankly I ran the numbers as I wrote it, and I must say I'm saddened by coming to a largely negative result.

If there's interest, I'll try and do another analysis piece speculating on the price of DOGE and our required hashrate through halvenings, but that's for another day.

Dogecoin part I

This post originally appeared on reddit.com/r/dogecoin, and was added to my official tech blog for posterity. or something.


WARNING: WALL OF TEXT, HIGH SCIENCE CONTENT

Friends, shibes, it is my pleasure to speak with you for what I hope is the first and not the last time. I'm /u/arrdem [1] , I'm a Doge daytrader, economist and miner on the side, and a programmer during the day. Today I'd like to have a chat about some of the rumors with regards to ASICs and the X11 hash that have been floating around /r/dogecoin[2] for the last few weeks and I hope bring some light to the discussions.

On Scrypt

What is special about our hash function? Why does Bitcoin use SHA256 and why does Doge use Scrypt? The hash function used by each cryptocurrency must have no known inverse function or algorithmic weakness which allowing miners to cheat and compute nonces easily, and it needs to be easy to verify or recompute given an input. The first requirement is obvious in that if the hash function is weak, then someone can achieve a 51% attack potentially with less than 51% of the network's hashing power. The second is less obvious and is in fact entirely a performance issue.

SHA256 is a known and trusted algorithm which has yet to exhibit any known weaknesses, and it is very very fast to recompute. This is why Bitcoin is SHA based.

Litecoin, the intellectual father of Dogecoin, chose the Scrypt hash function because it was a memory bound algorithm. That is, the slowest part of computing the Scrypt hash of some value is waiting for values to be fetched from memory: an operation which it is amazingly expensive to make fast. The goal of choosing an artificially expensive hash function was to escape the Application Specific Integrated Circuits (ASICs or hardware miners) which had come to dominate Bitcoin mining. Because the SHA256 algorithm does not have large memory requirements, it was easy for Bitcoin speculators to develop cost effective hardware for the single purpose of searching for SHA256 nonce values.

On ASICs

Before we get to whether ASICs are good or bad for a coin, we must first assess why they made sense for Bitcoin so that we can reason about their impact on Doge.

Because the computational power to find a nonce for any good cryptocurrency is expected to be large, that means there is a literal cost attached to processing each transaction on the network. While transactions may be nominally free or at least low fee, miners are really speculators expecting that someday the value of the coins they earn computing nonce values for blocks will exceed the operating costs and purchase costs of the hardware they mine with. This expectation that one day mining costs will be repaid is in fact the key reason that Bitcoin featured block rewards. The block reward was seen as a bootstrapping mechanic with which to buy hardware investment in the Bitcoin network through currency inflation.

Now, ASICs and other mining hardware only pay for themselves if one expects to get enough return from block rewards and future coin price increases to cover the purchase and operating costs of the hardware. However, this is where the block schedule comes in. If we expect that thanks to the law of large numbers that one's return is on average the block reward times ones fraction of the network hashrate, it becomes clear that as the block reward falls it becomes very difficult for any purchased mining hardware to pay itself off let alone turn a profit.

On the block schedule

Looking at the Bitcoin block schedule, ASICs kinda make sense. The Bitcoin block schedule extends until 2140, at which time the "omega block" will be mined and the per block reward of Bitcoin mining will become zero. However until that time the per block reward will decrease 50% every four years. Today in 2014, the per block reward of Bitcoin is 25BTC and it won't change until 2017. That means that Bitcoin targeted ASICs can potentially run for three whole years or more and still have a reasonable chance of breaking even with no assuptions made about changes in the value of 1BTC.

Doge's block schedule looks completely different. Where Bitcoin has a long tail on its per block reward extending out to 2140, Dogecoin will reach it's minimum block reward at block 600,000 in January of 2015, less than 14 months after Dogecoin came into being. With the 3rd halvening about 11 days out and the 4th on the horizon, by the time big boy ASICs for Scrypt start shipping in Q3/Q4, being September and later, the per block reward of Doge will have fallen to 31.25KDOGE and below. Third generation ASICs slated for December and January will likely never see more than 15.625KDOGE/block.

On the price of Doge

So what does this mean for the price of Doge? If the price of Doge doesn't increase at all, it's clear that the expensive new ASICs will never break even. This suggests that late comers with high powered mining hardware will be looking to recoup their investments and asking higher and higher prices for their Doge which should drive up the price overall.

To put some numbers on this, at current prices and hashrate, accounting for halvenings, neither Gridseed ASIC even breaks even within 200 days if purchased within the next 48hrs. fn:1. Wait 30 days (after the comming halvening) and you don't come anywhere near break even. If I change my model to include some hashrate growth factor, the outlook is even worse. fn:2.

This isn't bad news. This is awesome news for the price of DOGE. Lets say that Gridseed ships oh 500 units of their big boy ASIC, which may be conservative. fn:3 That's right, if hardware equivalent to 1K large Gridseeds came on in the next 30 days and ran at least for 200, doge would have to go all the way up to 702DOGE/USD just for them to break even!

To the moon

So where does this leave us. I think that the numbers I've presented here show that ASICs for Dogecoin are patently absurd, unless you expect to see a gargantuan spike in the price of DOGE which would make us all rich men anyway. While I'm willing to speculate on block reward (which is easy to model) and on hashrate which I assume is more or less linear, I have no mechanism with which I can confidently predict the price of DOGE out more than a week. Naive linear projections from our initial open of 80 satoshi to today's 126 satoshi over the course of four months suggests that in 200 days we could well see the ~300 satoshi prices which would make Gridseed and other ASIC miners profitable. However once you account for the high volatility of Doge, of Bitcoin and general market manipulation who knows if it'd ever go that high stably.

So. To sum up. On the basis of these sketchy ROI numbers, I think that buying ASICs is probably ill advised. That said, I expect that people will buy ASICs and that in doing so they will drive up the price of DOGE at the same time as the supply of DOGE starts to dry up due to block reward decreases.

I will be interested to see what happens to DOGE mining in January, as we will be the first coin to reach their steady mining state. I hope that the 10,000 DOGE reward per block will be sufficient to support the ASIC and GPU mining required to keep our hashrate out of 51% threat, but only time will tell. There is a real threat that the ROI of mining will be too low to justify the purchase of new ASIC let alone GPU hardware, which would lead to a falling hashrate and a credible threat of 51% vulnerability. However we could also see prices to go to the moon in which case that is no worry as high efficiency ASIC farms would take over mining securing the coin's stability more or less. I will note that no coin has yet solved the 51% threat issues posed by centralized mining, and I'm personally convinced that it's an intractable problem because as rewards per block decrease as for bitcoin, the costs of mining operations must likewise fall leading to greater centralization of compute power. By fixing our block reward we may. may. be able to dodge (ha ha) this issue however the essential drive to cut mining prices for ROI maximization will remain and will continue to drive mining centralization.

With all this in mind, it's silly to talk about the adoption of X11 or another hashing algorithm, because if and when ASIC miners for DOGE become big business it'll already be too late and we will have already mined the vast majority of DOGE thus securing the distribution of DOGE away from the ASIC miners we seem to fear so much as a community. Making the switch to X11 simply delays the ASIC hardware which we want anyway due to the price increases it's likely to drive, forget about making us artificially dependent on GPU mining to secure our hashrate and creating an uncalled for blockchain fork.

TL;DR

  • Stop worrying and love the ASICs, they won't make a ton of money and will secure our hashrate and by proxy our Doges!
  • STFU about X11. It's even more ASIC friendly than Scrypt, and we gain nothing from another blockchain fork.
  • Price projection: moon!
  • Open issue: How do we limit mining centralization without increasing inflation? Are we already at a balance point?

MSC

The software I've built and used to make these models is entirely open source and written in Clojure, see the footnotes for source and libraries.

Other programs involved

https://www.refheap.com/78314

https://github.com/arrdem/meajure

Edit History

  1. Wording typo fixed
  2. Fix fn:2 to reflect increased network hashrate
  3. Don't bother asking me what I think the price of DOGE will be. Not the foggiest.
  4. Fix final block reward, 10k not 100k
  5. Fix omega block date for BTC, 2140 not 2024