Bitcoin network simulation testing?
5 messages, 4 to 9 Oct 2015. Started by Byron Gibson.
Byron Gibson (2), naama.kates, Andrew Miller, Pindar Wong
Hi all, is anyone using simulators like Shadow (https://shadow.github.io), BTCSim (https://github.com/btcsuite/btcsim), etc. to test proposed changes to Bitcoin? I have a few questions about their capabilities and limitations.
Byron Gibson http://mirror.co/ https://keybase.io/byrongibson
Hi Byron,
I've been using shadow a bit-- I think these simulators are important for testing, but Shadow, at least, certainly seems to have limitations, in some crucial respects. Running shadow w Tor (which is only logical, because many BCT transactions transpire over Tor) is not as 'light' as presented and slows my own box down quite a bit, so the stats can't possibly be accurate... I don't know if this answers any questions or if you've had this experience at all -- perhaps it is negligible on a more powerful machine than my own-- or perhaps there is an adjustment still unaccounted?
Regards, Nina K
Sent from my iPhone
Quoted text (14 lines)
> On Oct 4, 2015, at 2:04 PM, Byron Gibson via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote: > > Hi all, is anyone using simulators like Shadow (https://shadow.github.io), BTCSim (https://github.com/btcsuite/btcsim), etc. to test proposed changes to Bitcoin? I have a few questions about their capabilities and limitations. > > Byron Gibson > http://mirror.co/ > https://keybase.io/byrongibson > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Shadow uses virtual time, entirely decoupled from real time. So while it may slow down your machine, this would not affect the stats collected
(although it does make shadow somewhat unpleasant to run, unless you have a fast machine, compared to abstract simulators that avoid running the actual Bitcoin code). Hi Byron,
I've been using shadow a bit-- I think these simulators are important for testing, but Shadow, at least, certainly seems to have limitations, in some crucial respects. Running shadow w Tor (which is only logical, because many BCT transactions transpire over Tor) is not as 'light' as presented and slows my own box down quite a bit, so the stats can't possibly be accurate... I don't know if this answers any questions or if you've had this experience at all -- perhaps it is negligible on a more powerful machine than my own-- or perhaps there is an adjustment still unaccounted?
Regards, Nina K
Sent from my iPhone
On Oct 4, 2015, at 2:04 PM, Byron Gibson via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Hi all, is anyone using simulators like Shadow (https://shadow.github.io), BTCSim (https://github.com/btcsuite/btcsim), etc. to test proposed changes to Bitcoin? I have a few questions about their capabilities and limitations.
Byron Gibson http://mirror.co/ https://keybase.io/byrongibson
Thanks Nina and Andrew, I may have the capability to run a simulator large scale on many nodes on AWS/GCE, but am looking to optimize the ROI by selecting categories of experiments that benefit from a more controlled environment, rather than be invalidated by its divergence/s from livenet. But wondering if anyone else has already been doing this, would be nice to collaborate.
Byron Gibson CoS | http://mirrorx.com/ https://onename.io/bgibson https://keybase.io/byrongibson
On Oct 6, 2015 14:00, "Andrew Miller" <amiller@cs•umd•edu> wrote:
Quoted text (48 lines)
> Shadow uses virtual time, entirely decoupled from real time. So while it > may slow down your machine, this would not affect the stats collected > > (although it does make shadow somewhat unpleasant to run, unless you have > a fast machine, compared to abstract simulators that avoid running the > actual Bitcoin code). > Hi Byron, > > I've been using shadow a bit-- I think these simulators are important for > testing, but Shadow, at least, certainly seems to have limitations, in some > crucial respects. Running shadow w Tor (which is only logical, because > many BCT transactions transpire over Tor) is not as 'light' as presented > and slows my own box down quite a bit, so the stats can't possibly be > accurate... I don't know if this answers any questions or if you've had > this experience at all -- perhaps it is negligible on a more powerful > machine than my own-- or perhaps there is an adjustment still unaccounted? > > Regards, > Nina K > > Sent from my iPhone > > On Oct 4, 2015, at 2:04 PM, Byron Gibson via bitcoin-dev < > bitcoin-dev@lists•linuxfoundation•org> wrote: > > Hi all, is anyone using simulators like Shadow (https://shadow.github.io), > BTCSim (https://github.com/btcsuite/btcsim), etc. to test proposed > changes to Bitcoin? I have a few questions about their capabilities and > limitations. > > Byron Gibson > http://mirror.co/ > https://keybase.io/byrongibson > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >
Byron: I'd be interested to collaborate on this in some fashion.
p.
On Sat, Oct 10, 2015 at 1:11 AM, Byron Gibson via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (69 lines)
> Thanks Nina and Andrew, I may have the capability to run a simulator large > scale on many nodes on AWS/GCE, but am looking to optimize the ROI by > selecting categories of experiments that benefit from a more controlled > environment, rather than be invalidated by its divergence/s from livenet. > But wondering if anyone else has already been doing this, would be nice to > collaborate. > > Byron Gibson > CoS | http://mirrorx.com/ > https://onename.io/bgibson > https://keybase.io/byrongibson > > > On Oct 6, 2015 14:00, "Andrew Miller" <amiller@cs•umd•edu> wrote: > >> Shadow uses virtual time, entirely decoupled from real time. So while it >> may slow down your machine, this would not affect the stats collected >> >> (although it does make shadow somewhat unpleasant to run, unless you have >> a fast machine, compared to abstract simulators that avoid running the >> actual Bitcoin code). >> Hi Byron, >> >> I've been using shadow a bit-- I think these simulators are important for >> testing, but Shadow, at least, certainly seems to have limitations, in some >> crucial respects. Running shadow w Tor (which is only logical, because >> many BCT transactions transpire over Tor) is not as 'light' as presented >> and slows my own box down quite a bit, so the stats can't possibly be >> accurate... I don't know if this answers any questions or if you've had >> this experience at all -- perhaps it is negligible on a more powerful >> machine than my own-- or perhaps there is an adjustment still unaccounted? >> >> Regards, >> Nina K >> >> Sent from my iPhone >> >> On Oct 4, 2015, at 2:04 PM, Byron Gibson via bitcoin-dev < >> bitcoin-dev@lists•linuxfoundation•org> wrote: >> >> Hi all, is anyone using simulators like Shadow (https://shadow.github.io), >> BTCSim (https://github.com/btcsuite/btcsim), etc. to test proposed >> changes to Bitcoin? I have a few questions about their capabilities and >> limitations. >> >> Byron Gibson >> http://mirror.co/ >> https://keybase.io/byrongibson >> >> >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation•org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation•org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >> > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >