Area of Focus
9 messages, 20 Dec 2014. Started by Will Bickford.
Will Bickford (2), Matt Corallo (2), Roy Badami, Jeremy Spilman, Christian Decker, Jeff Garzik, Greg Maxwell
Hi all, I'm looking to help with Bitcoin core development in my spare time (a few hours per week).
A little bit about me: * I use C++ and Qt daily * I love to automate and enhance software systems * I enjoy root causing and fixing issues
I saw Gavin say we needed help with testing in a Reddit AMA a while ago. I'm curious where I can make the best impact. Any feedback would be appreciated. Thanks!
Will Bickford "In Google We Trust"
There was recently some discussion around dnsseeds. Currently some dnsseeds are getting blocked by ISPs because the hosts they pick up (which run bitcoin core nodes) often run rather web servers alongside which serve malware or whatever else and thus end up on IP-based malware blacklists.
Of course we really dont want to move off of DNS because it has this big built-in anonymity network where the DNS seed servers only get information about your ISP, not you, and its cached so you dont get as much information about how many users are making those requests.
A potential solution might be supporting some subdomain which has results XORed with some constant mask to tweak the real IP.
Additionally, it might be cool to stuff a TXT/AAAA/whatever record with a signature of the results provided by the DNSseed operator.
Matt
On 12/20/14 07:42, Will Bickford wrote:
Quoted text (30 lines)
> Hi all, I'm looking to help with Bitcoin core development in my spare > time (a few hours per week). > > A little bit about me: > * I use C++ and Qt daily > * I love to automate and enhance software systems > * I enjoy root causing and fixing issues > > I saw Gavin say we needed help with testing in a Reddit AMA a while ago. > I'm curious where I can make the best impact. Any feedback would be > appreciated. Thanks! > > Will Bickford > "In Google We Trust" > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
Why would we want to have anything to do with people who are hosting malware? Or do I misunderstand?
On Sat, Dec 20, 2014 at 08:57:53AM +0000, Matt Corallo wrote:
Quoted text (62 lines)
> There was recently some discussion around dnsseeds. Currently some > dnsseeds are getting blocked by ISPs because the hosts they pick up > (which run bitcoin core nodes) often run rather web servers alongside > which serve malware or whatever else and thus end up on IP-based malware > blacklists. > > Of course we really dont want to move off of DNS because it has this big > built-in anonymity network where the DNS seed servers only get > information about your ISP, not you, and its cached so you dont get as > much information about how many users are making those requests. > > A potential solution might be supporting some subdomain which has > results XORed with some constant mask to tweak the real IP. > > Additionally, it might be cool to stuff a TXT/AAAA/whatever record with > a signature of the results provided by the DNSseed operator. > > Matt > > On 12/20/14 07:42, Will Bickford wrote: > > Hi all, I'm looking to help with Bitcoin core development in my spare > > time (a few hours per week). > > > > A little bit about me: > > * I use C++ and Qt daily > > * I love to automate and enhance software systems > > * I enjoy root causing and fixing issues > > > > I saw Gavin say we needed help with testing in a Reddit AMA a while ago. > > I'm curious where I can make the best impact. Any feedback would be > > appreciated. Thanks! > > > > Will Bickford > > "In Google We Trust" > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration & more > > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > > > > > _______________________________________________ > > Bitcoin-development mailing list > > Bitcoin-development@lists•sourceforge•net > > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
On Sat, Dec 20, 2014 at 08:57:53AM +0000, Matt Corallo wrote:
Quoted text (5 lines)
>> There was recently some discussion around dnsseeds. Currently some >> dnsseeds are getting blocked by ISPs because the hosts they pick up >> (which run bitcoin core nodes) often run rather web servers alongside >> which serve malware or whatever else and thus end up on IP-based malware >> blacklists.
On Sat, 20 Dec 2014 02:08:17 -0800, Roy Badami <roy@gnomon•org•uk> wrote:
Quoted text (2 lines)
> Why would we want to have anything to do with people who are hosting > malware? Or do I misunderstand?
It sounds like Matt is saying the nodes the dnsseed is pointing to as valid full nodes, that those IPs are hosting the malware. Since the dnsseed picks up any stable nodes it can find without auditing, it's perhaps not surprising some servers in the world are running a full node and a malware server together.
I guess what confused me about this though, how are ISPs reading the dnsseed's node list, scanning *those* IPs for malware, and then ending up blocking the dnsseed? Seems like a pretty winding path to end up blocking a DNS server?
Since when do ISPs null-route a DNS server for happening to resolve some domains to IPs which happen to also be hosting some malware? Null-route those endpoint IPs sure, but the DNS server too? I guess there was that incident of Microsoft taking over No-IP.com -- are dnsseeds being blocked ostensibly because they are acting as dyanamic DNS infrastructure for malware sites?
Thanks for bringing this to my attention.
I added a safety check to my crawler and seed.bitcoinstats.com should not return IPs that also run HTTP or HTTPS, hopefully this'll keep it off blacklists :-)
Well, some ISPs, when they see an IP address serving malware, will (apparently) simply replace DNS results for anything returning that IP with a warning page.
One solutions is to just blindly block everything with HTTP(S), as Christian has done, but this is a rather ugly solution, since many perfectly good nodes will get caught in the crossfire. Hiding what actual IPs we're returning in the results seems much cleaner, despite being an ugly hack.
On 12/20/14 11:14, Jeremy Spilman wrote:
Quoted text (41 lines)
> On Sat, Dec 20, 2014 at 08:57:53AM +0000, Matt Corallo wrote: >>> There was recently some discussion around dnsseeds. Currently some >>> dnsseeds are getting blocked by ISPs because the hosts they pick up >>> (which run bitcoin core nodes) often run rather web servers alongside >>> which serve malware or whatever else and thus end up on IP-based malware >>> blacklists. > > On Sat, 20 Dec 2014 02:08:17 -0800, Roy Badami <roy@gnomon•org•uk> wrote: >> Why would we want to have anything to do with people who are hosting >> malware? Or do I misunderstand? > > It sounds like Matt is saying the nodes the dnsseed is pointing to as > valid full nodes, that those IPs are hosting the malware. Since the > dnsseed picks up any stable nodes it can find without auditing, it's > perhaps not surprising some servers in the world are running a full node > and a malware server together. > > I guess what confused me about this though, how are ISPs reading the > dnsseed's node list, scanning *those* IPs for malware, and then ending up > blocking the dnsseed? Seems like a pretty winding path to end up blocking > a DNS server? > > Since when do ISPs null-route a DNS server for happening to resolve some > domains to IPs which happen to also be hosting some malware? Null-route > those endpoint IPs sure, but the DNS server too? I guess there was that > incident of Microsoft taking over No-IP.com -- are dnsseeds being blocked > ostensibly because they are acting as dyanamic DNS infrastructure for > malware sites? > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
Getting back to the original topic...
I would recommend first taking a look at how the current tests are built (via autoconf/automake) in src/test. There are several surfaces to test, RPC, REST, P2P, internal unit tests, and more. Then, Travis applies a second level of testing via the bitcoinj-based regression tests.
Some automated tests that operate at the Qt level would be interesting. In general, the current tests only scratch the surface of what Needs To Be Tested... but part of figuring out a good test is (a) knowing bitcoin and (b) knowing the current test regimes.
Join #bitcoin-dev IRC and ask questions. Read the bitcoin wiki.
On Sat, Dec 20, 2014 at 2:42 AM, Will Bickford <wbic16@gmail•com> wrote:
Quoted text (30 lines)
> > Hi all, I'm looking to help with Bitcoin core development in my spare time > (a few hours per week). > > A little bit about me: > * I use C++ and Qt daily > * I love to automate and enhance software systems > * I enjoy root causing and fixing issues > > I saw Gavin say we needed help with testing in a Reddit AMA a while ago. > I'm curious where I can make the best impact. Any feedback would be > appreciated. Thanks! > > Will Bickford > "In Google We Trust" > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > >
On Sat, Dec 20, 2014 at 11:14 AM, Jeremy Spilman <jeremy@taplink•co> wrote:
Quoted text (3 lines)
>> are dnsseeds being blocked > ostensibly because they are acting as dyanamic DNS infrastructure for > malware sites?
Pretty much appears to be the case. In every instance it appears to be automated. This predates the msft no-ip.com stuff. We also had similar problems with the IRC based method that the software originally used.
It's the same story for mail relay spam blacklisting. There is a whole industry out there selling people semi-snake-oil blocking solutions to make the baddness of the internet go away. The low margin business demands a cheap and highly automated approach... lots of inappropriate things get blocked. Nagging people to fix things is time consuming, better to move out of their sights a bit, so that they at least have to specifically target Bitcoin. If they do, it'll at least be worth the time spent fixing it.
I believe opendns is blocking all of sipa.be still as we speak, so if you'd like to see it for yourself try to load http://bitcoin.sipa.be while using opendns.
Thanks Jeff. I'll start looking there.
Will Bickford "In Google We Trust"
On Sat, Dec 20, 2014 at 3:26 PM, Jeff Garzik <jgarzik@bitpay•com> wrote:
Quoted text (54 lines)
> Getting back to the original topic... > > I would recommend first taking a look at how the current tests are built > (via autoconf/automake) in src/test. There are several surfaces to test, > RPC, REST, P2P, internal unit tests, and more. Then, Travis applies a > second level of testing via the bitcoinj-based regression tests. > > Some automated tests that operate at the Qt level would be interesting. > In general, the current tests only scratch the surface of what Needs To Be > Tested... but part of figuring out a good test is (a) knowing bitcoin and > (b) knowing the current test regimes. > > Join #bitcoin-dev IRC and ask questions. Read the bitcoin wiki. > > > > > On Sat, Dec 20, 2014 at 2:42 AM, Will Bickford <wbic16@gmail•com> wrote: > >> Hi all, I'm looking to help with Bitcoin core development in my spare >> time (a few hours per week). >> >> A little bit about me: >> * I use C++ and Qt daily >> * I love to automate and enhance software systems >> * I enjoy root causing and fixing issues >> >> I saw Gavin say we needed help with testing in a Reddit AMA a while ago. >> I'm curious where I can make the best impact. Any feedback would be >> appreciated. Thanks! >> >> Will Bickford >> "In Google We Trust" >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists•sourceforge•net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> >> > > -- > Jeff Garzik > Bitcoin core developer and open source evangelist > BitPay, Inc. https://bitpay.com/ >