CI Build for Bitcoin - Some Basic Questions about Gitian and other stuff
6 messages, 23 to 24 Sep 2015. Started by Roy Osherove.
Roy Osherove (3), Mark F (2), Jonas Schnelli
Hi Folks. I'm trying my hand at creating a reproducible build of my own for bitcoin and bitcoin-XT, using TeamCity. I believe it is the best way to learn something: To try to build it yourself. Here is what I think I know so far, and I would love corrections, plus questions:
1. Bitcoin is built continuously on travis-CI at https://travis-ci.org/bitcoin/bitcoin/ 2. there are many flavors that are built, but I'm not sure if all of them are actually used/necessary. are they all needed, or just to "just in case"? 3. There is a gitian build file for bitcoin, but is anyone actually using it? are the bin files on bitcoin.org taken from that? or the travis ci builds? or some other place? 4. Are there any things that people would love to have in the build that do not exist there today? perhaps I can help with that?
Here is what I have now: http://btcdev.osherove.com:8111/ It does not do the matrix build yet, but it's coming. I'm just wondering if all the platforms need to be supported,and if gitian is truly required to be used, or used in parallel, or at all..
Feedback appreciated.
The builds made by Travis are for the purpose of making sure that the source code compiles and tests run successfully on all supported platforms. The binaries are not used anywhere else because Travis is not a trusted platform.
The binaries on bitcoin.org are built using the gitian process and signed by a quorum of developers.
On Wed, Sep 23, 2015 at 10:13 AM, Roy Osherove via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (46 lines)
> Hi Folks. > I'm trying my hand at creating a reproducible build of my own for bitcoin > and bitcoin-XT, using TeamCity. > I believe it is the best way to learn something: To try to build it > yourself. > Here is what I think I know so far, and I would love corrections, plus > questions: > > 1. Bitcoin is built continuously on travis-CI at > https://travis-ci.org/bitcoin/bitcoin/ > 2. there are many flavors that are built, but I'm not sure if all of > them are actually used/necessary. are they all needed, or just to "just in > case"? > 3. There is a gitian build file for bitcoin, but is anyone actually > using it? are the bin files on bitcoin.org taken from that? or the > travis ci builds? or some other place? > 4. Are there any things that people would love to have in the build > that do not exist there today? perhaps I can help with that? > > Here is what I have now: http://btcdev.osherove.com:8111/ > It does not do the matrix build yet, but it's coming. I'm just wondering > if all the platforms need to be supported,and if gitian is truly required > to be used, or used in parallel, or at all.. > > Feedback appreciated. > > -- > Thanks, > > Roy Osherove > > - *@RoyOsherove* <https://twitter.com/RoyOsherove> > - Read my new book *Notes to a Software Team Leader > <http://TeamLeadSkills.com>* > - Or *my new course* about Beautiful Builds > <http://courses.osherove.com> and Continuous Delivery > - +1-201-256-5575 > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >
Thanks Mark. Is there a public server where the gitian builds can be viewed? Is there a public server that shows the quorum verifications or that shows how to join in on the verification if such as thing is helpful?
On Wed, Sep 23, 2015 at 10:18 AM, Mark Friedenbach <mark@friedenbach•org> wrote:
Quoted text (58 lines)
> The builds made by Travis are for the purpose of making sure that the > source code compiles and tests run successfully on all supported platforms. > The binaries are not used anywhere else because Travis is not a trusted > platform. > > The binaries on bitcoin.org are built using the gitian process and signed > by a quorum of developers. > > On Wed, Sep 23, 2015 at 10:13 AM, Roy Osherove via bitcoin-dev < > bitcoin-dev@lists•linuxfoundation•org> wrote: > >> Hi Folks. >> I'm trying my hand at creating a reproducible build of my own for bitcoin >> and bitcoin-XT, using TeamCity. >> I believe it is the best way to learn something: To try to build it >> yourself. >> Here is what I think I know so far, and I would love corrections, plus >> questions: >> >> 1. Bitcoin is built continuously on travis-CI at >> https://travis-ci.org/bitcoin/bitcoin/ >> 2. there are many flavors that are built, but I'm not sure if all of >> them are actually used/necessary. are they all needed, or just to "just in >> case"? >> 3. There is a gitian build file for bitcoin, but is anyone actually >> using it? are the bin files on bitcoin.org taken from that? or the >> travis ci builds? or some other place? >> 4. Are there any things that people would love to have in the build >> that do not exist there today? perhaps I can help with that? >> >> Here is what I have now: http://btcdev.osherove.com:8111/ >> It does not do the matrix build yet, but it's coming. I'm just wondering >> if all the platforms need to be supported,and if gitian is truly required >> to be used, or used in parallel, or at all.. >> >> Feedback appreciated. >> >> -- >> Thanks, >> >> Roy Osherove >> >> - *@RoyOsherove* <https://twitter.com/RoyOsherove> >> - Read my new book *Notes to a Software Team Leader >> <http://TeamLeadSkills.com>* >> - Or *my new course* about Beautiful Builds >> <http://courses.osherove.com> and Continuous Delivery >> - +1-201-256-5575 >> >> >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation•org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >> >
Well the gitian builds are made available on bitcoin.org. If you mean a build server where gitian builds are automatically done and made available, well that rather defeats the point of gitian.
The quorum signatures are accumulated here: https://github.com/bitcoin/gitian.sigs (it's a manual process).
On Wed, Sep 23, 2015 at 10:31 AM, Roy Osherove <roy@osherove•com> wrote:
Quoted text (83 lines)
> Thanks Mark. > Is there a public server where the gitian builds can be viewed? > Is there a public server that shows the quorum verifications or that shows > how to join in on the verification if such as thing is helpful? > > On Wed, Sep 23, 2015 at 10:18 AM, Mark Friedenbach <mark@friedenbach•org> > wrote: > >> The builds made by Travis are for the purpose of making sure that the >> source code compiles and tests run successfully on all supported platforms. >> The binaries are not used anywhere else because Travis is not a trusted >> platform. >> >> The binaries on bitcoin.org are built using the gitian process and >> signed by a quorum of developers. >> >> On Wed, Sep 23, 2015 at 10:13 AM, Roy Osherove via bitcoin-dev < >> bitcoin-dev@lists•linuxfoundation•org> wrote: >> >>> Hi Folks. >>> I'm trying my hand at creating a reproducible build of my own for >>> bitcoin and bitcoin-XT, using TeamCity. >>> I believe it is the best way to learn something: To try to build it >>> yourself. >>> Here is what I think I know so far, and I would love corrections, plus >>> questions: >>> >>> 1. Bitcoin is built continuously on travis-CI at >>> https://travis-ci.org/bitcoin/bitcoin/ >>> 2. there are many flavors that are built, but I'm not sure if all >>> of them are actually used/necessary. are they all needed, or just to "just >>> in case"? >>> 3. There is a gitian build file for bitcoin, but is anyone actually >>> using it? are the bin files on bitcoin.org taken from that? or the >>> travis ci builds? or some other place? >>> 4. Are there any things that people would love to have in the build >>> that do not exist there today? perhaps I can help with that? >>> >>> Here is what I have now: http://btcdev.osherove.com:8111/ >>> It does not do the matrix build yet, but it's coming. I'm just wondering >>> if all the platforms need to be supported,and if gitian is truly required >>> to be used, or used in parallel, or at all.. >>> >>> Feedback appreciated. >>> >>> -- >>> Thanks, >>> >>> Roy Osherove >>> >>> - *@RoyOsherove* <https://twitter.com/RoyOsherove> >>> - Read my new book *Notes to a Software Team Leader >>> <http://TeamLeadSkills.com>* >>> - Or *my new course* about Beautiful Builds >>> <http://courses.osherove.com> and Continuous Delivery >>> - +1-201-256-5575 >>> >>> >>> >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists•linuxfoundation•org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >>> >> > > > -- > Thanks, > > Roy Osherove > > - *@RoyOsherove* <https://twitter.com/RoyOsherove> > - Read my new book *Notes to a Software Team Leader > <http://TeamLeadSkills.com>* > - Or *my new course* about Beautiful Builds > <http://courses.osherove.com> and Continuous Delivery > - +1-201-256-5575 > - Timezone: Eastern Standard Time (New York) > > >
So who physically manually uploads the gitian build to bitcoin.org?
On Wed, Sep 23, 2015 at 11:10 AM, Mark Friedenbach <mark@friedenbach•org> wrote:
Quoted text (93 lines)
> Well the gitian builds are made available on bitcoin.org. If you mean a > build server where gitian builds are automatically done and made available, > well that rather defeats the point of gitian. > > The quorum signatures are accumulated here: > https://github.com/bitcoin/gitian.sigs (it's a manual process). > > On Wed, Sep 23, 2015 at 10:31 AM, Roy Osherove <roy@osherove•com> wrote: > >> Thanks Mark. >> Is there a public server where the gitian builds can be viewed? >> Is there a public server that shows the quorum verifications or that >> shows how to join in on the verification if such as thing is helpful? >> >> On Wed, Sep 23, 2015 at 10:18 AM, Mark Friedenbach <mark@friedenbach•org> >> wrote: >> >>> The builds made by Travis are for the purpose of making sure that the >>> source code compiles and tests run successfully on all supported platforms. >>> The binaries are not used anywhere else because Travis is not a trusted >>> platform. >>> >>> The binaries on bitcoin.org are built using the gitian process and >>> signed by a quorum of developers. >>> >>> On Wed, Sep 23, 2015 at 10:13 AM, Roy Osherove via bitcoin-dev < >>> bitcoin-dev@lists•linuxfoundation•org> wrote: >>> >>>> Hi Folks. >>>> I'm trying my hand at creating a reproducible build of my own for >>>> bitcoin and bitcoin-XT, using TeamCity. >>>> I believe it is the best way to learn something: To try to build it >>>> yourself. >>>> Here is what I think I know so far, and I would love corrections, plus >>>> questions: >>>> >>>> 1. Bitcoin is built continuously on travis-CI at >>>> https://travis-ci.org/bitcoin/bitcoin/ >>>> 2. there are many flavors that are built, but I'm not sure if all >>>> of them are actually used/necessary. are they all needed, or just to "just >>>> in case"? >>>> 3. There is a gitian build file for bitcoin, but is anyone >>>> actually using it? are the bin files on bitcoin.org taken from >>>> that? or the travis ci builds? or some other place? >>>> 4. Are there any things that people would love to have in the build >>>> that do not exist there today? perhaps I can help with that? >>>> >>>> Here is what I have now: http://btcdev.osherove.com:8111/ >>>> It does not do the matrix build yet, but it's coming. I'm just >>>> wondering if all the platforms need to be supported,and if gitian is truly >>>> required to be used, or used in parallel, or at all.. >>>> >>>> Feedback appreciated. >>>> >>>> -- >>>> Thanks, >>>> >>>> Roy Osherove >>>> >>>> - *@RoyOsherove* <https://twitter.com/RoyOsherove> >>>> - Read my new book *Notes to a Software Team Leader >>>> <http://TeamLeadSkills.com>* >>>> - Or *my new course* about Beautiful Builds >>>> <http://courses.osherove.com> and Continuous Delivery >>>> - +1-201-256-5575 >>>> >>>> >>>> >>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists•linuxfoundation•org >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>> >>>> >>> >> >> >> -- >> Thanks, >> >> Roy Osherove >> >> - *@RoyOsherove* <https://twitter.com/RoyOsherove> >> - Read my new book *Notes to a Software Team Leader >> <http://TeamLeadSkills.com>* >> - Or *my new course* about Beautiful Builds >> <http://courses.osherove.com> and Continuous Delivery >> - +1-201-256-5575 >> - Timezone: Eastern Standard Time (New York) >> >> >> >
Hi Roy
Quoted text (2 lines)
> So who physically manually uploads the gitian build to bitcoin.org > <http://bitcoin.org>?
It's done by the core devs (very likely the project maintainer). I think it doesn't matter who did the upload to bitcoin.org.
What really would matter is – if users deciding to run a pre-compiled version of bitcoin-core – that they verify the binary against the available gitian sigs.
Signatur repository: https://github.com/bitcoin/gitian.sigs
PGP Pubkeys (mostly also available over gpg key servers): https://github.com/bitcoin/bitcoin/tree/master/contrib/gitian-downloader
Instruction available here: https://www.reddit.com/r/Bitcoin/wiki/verifying_bitcoin_core
/jonas