Bitcoin Protocol Specification
19 messages, 26 Jun to 14 Jul 2014. Started by Krzysztof Okupski.
Krzysztof Okupski (4), Mike Hearn (4), sickpig@gmail.com (3), Isidor Zeuner, JMOlmos GMail, Chris D'Costa, Matt Whitlock, Aaron Voisine, +3 more
Dear all,
a while ago I've posted a work of mine, namely the Bitcoin Protocol Specification, that should by now ring a bell to some of you. Ever since I've received plenty of comments from Bitcoin Core developers, Bitcoin enthusiasts and many others. For the encouraging words and improvement proposals I'm very grateful. I tried to incorporate to the best of my abilities and am happy to present a revised version. As before, it can be found under:
http://enetium.com/resources/Bitcoin.pdf
I hope it will prove useful to the community and thank in advance for any further improvement proposals.
Kind regards, Krzysztof
Hello Krzysztof,
[...]
Quoted text (7 lines)
> As before, it can be found under: > > http://enetium.com/resources/Bitcoin.pdf > > I hope it will prove useful to the community and thank in advance > for any further improvement proposals. >
I think it's great work and provides a good reference for those who want to get some insight into Bitcoin's design.
Have you considered putting the document source under version control, which may facilitate tracking future protocol improvements in the document easily?
Best regards,
Isidor
And for translation's facility :P
2014-07-07 14:57 GMT-03:00 JMOlmos GMail <colonizare@gmail•com>:
Quoted text (42 lines)
> And for translation's facility :P > > > 2014-07-02 22:21 GMT-03:00 Isidor Zeuner <cryptocurrencies@quidecco•de>: > > Hello Krzysztof, >> >> [...] >> > As before, it can be found under: >> > >> > http://enetium.com/resources/Bitcoin.pdf >> > >> > I hope it will prove useful to the community and thank in advance >> > for any further improvement proposals. >> > >> >> I think it's great work and provides a good reference for those >> who want to get some insight into Bitcoin's design. >> >> Have you considered putting the document source under version control, >> which may facilitate tracking future protocol improvements in the >> document easily? >> >> Best regards, >> >> Isidor >> >> >> ------------------------------------------------------------------------------ >> Open source business process management suite built on Java and Eclipse >> Turn processes into business applications with Bonita BPM Community >> Edition >> Quickly connect people, data, and systems into organized workflows >> Winner of BOSSIE, CODIE, OW2 and Gartner awards >> http://p.sf.net/sfu/Bonitasoft >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists•sourceforge•net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> > >
This is probably the best, most complete resource available for those who don't want to (or don't know how to) wade through the code. Well done.
On 7 July 2014 19:57, JMOlmos GMail <colonizare@gmail•com> wrote:
Quoted text (61 lines)
> And for translation's facility :P > > > 2014-07-07 14:57 GMT-03:00 JMOlmos GMail <colonizare@gmail•com>: > >> And for translation's facility :P >> >> >> 2014-07-02 22:21 GMT-03:00 Isidor Zeuner <cryptocurrencies@quidecco•de>: >> >> Hello Krzysztof, >>> >>> [...] >>> > As before, it can be found under: >>> > >>> > http://enetium.com/resources/Bitcoin.pdf >>> > >>> > I hope it will prove useful to the community and thank in advance >>> > for any further improvement proposals. >>> > >>> >>> I think it's great work and provides a good reference for those >>> who want to get some insight into Bitcoin's design. >>> >>> Have you considered putting the document source under version control, >>> which may facilitate tracking future protocol improvements in the >>> document easily? >>> >>> Best regards, >>> >>> Isidor >>> >>> >>> ------------------------------------------------------------------------------ >>> Open source business process management suite built on Java and Eclipse >>> Turn processes into business applications with Bonita BPM Community >>> Edition >>> Quickly connect people, data, and systems into organized workflows >>> Winner of BOSSIE, CODIE, OW2 and Gartner awards >>> http://p.sf.net/sfu/Bonitasoft >>> _______________________________________________ >>> Bitcoin-development mailing list >>> Bitcoin-development@lists•sourceforge•net >>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >>> >> >> > > > ------------------------------------------------------------------------------ > Open source business process management suite built on Java and Eclipse > Turn processes into business applications with Bonita BPM Community Edition > Quickly connect people, data, and systems into organized workflows > Winner of BOSSIE, CODIE, OW2 and Gartner awards > http://p.sf.net/sfu/Bonitasoft > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > >
Is anyone working on a similar specification document for Satoshi's P2P protocol? I know how blocks and transactions are structured and verified, but I'm interested in knowing how they're communicated over the network.
I wrote down a really short description in code comments for breadwallet, based on what I figured out:
https://github.com/voisine/breadwallet/blob/master/BreadWallet/BRPeer.m#L318
Aaron Voisine breadwallet.com
On Tue, Jul 8, 2014 at 1:04 PM, Matt Whitlock <bip@mattwhitlock•name> wrote:
Quoted text (12 lines)
> Is anyone working on a similar specification document for Satoshi's P2P protocol? I know how blocks and transactions are structured and verified, but I'm interested in knowing how they're communicated over the network. > > ------------------------------------------------------------------------------ > Open source business process management suite built on Java and Eclipse > Turn processes into business applications with Bonita BPM Community Edition > Quickly connect people, data, and systems into organized workflows > Winner of BOSSIE, CODIE, OW2 and Gartner awards > http://p.sf.net/sfu/Bonitasoft > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
There's this (not sure how complete it is): https://en.bitcoin.it/wiki/Protocol_specification
Also there's a good introduction to technical details here:
https://bitcointalk.org/index.php?topic=375524.0
I hope that is useful.
On 7/8/14, Matt Whitlock <bip@mattwhitlock•name> wrote:
Quoted text (15 lines)
> Is anyone working on a similar specification document for Satoshi's P2P > protocol? I know how blocks and transactions are structured and verified, > but I'm interested in knowing how they're communicated over the network. > > ------------------------------------------------------------------------------ > Open source business process management suite built on Java and Eclipse > Turn processes into business applications with Bonita BPM Community Edition > Quickly connect people, data, and systems into organized workflows > Winner of BOSSIE, CODIE, OW2 and Gartner awards > http://p.sf.net/sfu/Bonitasoft > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
Dear all,
thank you for your words of encouragement. I'll upload the raw .tex files on github shortly. If you have any other proposals, please let me know. Especially if there are any suggestions on how the signature section can be revised.
More information on network functionality can be found on: https://en.bitcoin.it/wiki/Network
Details can be found here: https://en.bitcoin.it/wiki/Satoshi_Client_Block_Exchange https://en.bitcoin.it/wiki/Satoshi_Client_Transaction_Exchange https://en.bitcoin.it/wiki/Satoshi_Client_Initialization_and_Startup https://en.bitcoin.it/wiki/Satoshi_Client_Node_Discovery https://en.bitcoin.it/wiki/Satoshi_Client_Node_Connectivity https://en.bitcoin.it/wiki/Satoshi_Client_Sockets_and_Messages
Warm greetings, Krzysztof
On Tue, Jul 8, 2014 at 10:04 PM, Matt Whitlock <bip@mattwhitlock•name> wrote:
Quoted text (3 lines)
> Is anyone working on a similar specification document for Satoshi's P2P > protocol? I know how blocks and transactions are structured and verified, > but I'm interested in knowing how they're communicated over the network.
There's a high level guide here:
https://bitcoin.org/en/developer-guide#p2p-network
It's not a protocol specification however, more a lay-of-the-land type description. The protocol is not very complicated however. The wiki page describes it adequately. It's basically just a framing mechanism with excessively aggressive checksumming, a version handshake, an inventory mechanism to cut down on bandwidth usage, the alerts, and that's about it.
Dear all,
thank you for your invaluable feedback. As requested, the spec will from now on be under version control. It can be found under:
https://github.com/minium/Bitcoin-Spec
The old link to the PDF will be, just in case, kept updated as well.
Warm greetings, Krzysztof Okupski
On Mon, Jul 14, 2014 at 11:54 AM, Krzysztof Okupski <k.okupski@googlemail•com> wrote:
Quoted text (12 lines)
> Dear all, > > thank you for your invaluable feedback. As requested, the spec > will from now on be under version control. It can be found under: > > https://github.com/minium/Bitcoin-Spec > > The old link to the PDF will be, just in case, kept updated as well. > > > Warm greetings, > Krzysztof Okupski
many many thanks Krzysztof.
This was badly needed.
This is a way to get more people to understand the whole system, hence increasing the number of bitcoin devs on the long term.
Nice work, but please don't call it the "Bitcoin protocol spec". Your document is not a spec. It is an attempt to describe in English the Bitcoin protocol, but anyone who implemented it based on your description would get it wrong. For example you didn't mention the SIGHASH_SINGLE bug and many other important areas like the difficulty transitions are also left unspecified.
As a loose description of the protocol for newbies it's an invaluable resource and perhaps we should link to it from the developer guide. As something that claims to be a specification it is quite possibly dangerous - the only spec that matters is the C++ original.
On Mon, Jul 14, 2014 at 11:54 AM, Krzysztof Okupski < k.okupski@googlemail•com> wrote:
Quoted text (26 lines)
> Dear all, > > thank you for your invaluable feedback. As requested, the spec > will from now on be under version control. It can be found under: > > https://github.com/minium/Bitcoin-Spec > > The old link to the PDF will be, just in case, kept updated as well. > > > Warm greetings, > Krzysztof Okupski > > > ------------------------------------------------------------------------------ > Want fast and easy access to all the code in your enterprise? Index and > search up to 200,000 lines of code with a free copy of Black Duck® > Code Sight™ - the same software that powers the world's largest code > search on Ohloh, the Black Duck Open Hub! Try it now. > http://p.sf.net/sfu/bds > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > >
Quoted text (2 lines)
> As a loose description of the protocol for newbies it's an invaluable > resource and perhaps we should link to it from the developer guide.
It has already been linked from the developer guide for quite a while, under Additional Resources.
Wladimir
Ah, that's great. Still, it would be good to be careful with the word "specification".
On Mon, Jul 14, 2014 at 1:41 PM, Wladimir <laanwj@gmail•com> wrote:
Quoted text (8 lines)
> > As a loose description of the protocol for newbies it's an invaluable > > resource and perhaps we should link to it from the developer guide. > > It has already been linked from the developer guide for quite a while, > under Additional Resources. > > Wladimir >
On Mon, Jul 14, 2014 at 1:26 PM, Mike Hearn <mike@plan99•net> wrote:
Quoted text (2 lines)
> Nice work, but please don't call it the "Bitcoin protocol spec". Your > document is not a spec.
fair objection, fwiw.
Quoted text (5 lines)
> It is an attempt to describe in English the Bitcoin > protocol, but anyone who implemented it based on your description would get > it wrong. For example you didn't mention the SIGHASH_SINGLE bug and many > other important areas like the difficulty transitions are also left > unspecified.
at the end of the day it's opensource so it could always be improved via pull request on github.
Krzysztof, in that regard I would add a licence file to make integration of changes made by others smoother.
Quoted text (4 lines)
> As a loose description of the protocol for newbies it's an invaluable > resource and perhaps we should link to it from the developer guide. As > something that claims to be a specification it is quite possibly dangerous - > the only spec that matters is the C++ original.
just out of curiosity, do you think it will be possible to create any other proper protocol specifications rather than the C++ original?
Quoted text (15 lines)
> On Mon, Jul 14, 2014 at 11:54 AM, Krzysztof Okupski > <k.okupski@googlemail•com> wrote: >> >> Dear all, >> >> thank you for your invaluable feedback. As requested, the spec >> will from now on be under version control. It can be found under: >> >> https://github.com/minium/Bitcoin-Spec >> >> The old link to the PDF will be, just in case, kept updated as well. >> >> >> Warm greetings, >> Krzysztof Okupski
Quoted text (3 lines)
> > just out of curiosity, do you think it will be possible to create any > other proper protocol specifications rather than the C++ original?
Well it's a finite code base so yes, it should be possible.
The only problem is .... so far everyone who tried it, didn't succeed :) Heck even people who tried to reimplement it by reading the code keep getting subtle details wrong.
So it should definitely be possible one day, assuming Bitcoin doesn't become radically more complex, but it's a minefield.
I've renamed it to "Bitcoin Developer Specification" a little while ago. Maybe it should rather be named "Bitcoin Developer Reference"? Either way, creating a good description of Bitcoin is an incremental process and there are certainly many quirks I'm not aware of. I hope that together we will soon be able to fill in the missing gaps.
Warm greetings, Krzysztof Okupski
On 07/14/2014 01:26 PM, Mike Hearn wrote:
Quoted text (11 lines)
> Nice work, but please don't call it the "Bitcoin protocol spec". Your > document is not a spec. It is an attempt to describe in English the Bitcoin > protocol, but anyone who implemented it based on your description would get > it wrong. For example you didn't mention the SIGHASH_SINGLE bug and many > other important areas like the difficulty transitions are also left > unspecified. > > As a loose description of the protocol for newbies it's an invaluable > resource and perhaps we should link to it from the developer guide. As > something that claims to be a specification it is quite possibly dangerous > - the only spec that matters is the C++ original.
On Mon, Jul 14, 2014 at 10:44 AM, Krzysztof Okupski <k.okupski@googlemail•com> wrote:
Quoted text (5 lines)
> I've renamed it to "Bitcoin Developer Specification" a little while ago. > Maybe it should rather be named "Bitcoin Developer Reference"? Either > way, creating a good description of Bitcoin is an incremental process > and there are certainly many quirks I'm not aware of. I hope that > together we will soon be able to fill in the missing gaps.
Firstly, it is an excellent document, and it should be useful in educating others.
I do agree that "specification" is not a good word to use.
On Mon, Jul 14, 2014 at 2:37 PM, Mike Hearn <mike@plan99•net> wrote:
Quoted text (13 lines)
> just out of curiosity, do you think it will be possible to create any >> other proper protocol specifications rather than the C++ original? > > > Well it's a finite code base so yes, it should be possible. > > The only problem is .... so far everyone who tried it, didn't succeed :) > Heck even people who tried to reimplement it by reading the code keep > getting subtle details wrong. > > So it should definitely be possible one day, assuming Bitcoin doesn't > become radically more complex, but it's a minefield. >
thank for the frank reply