Future of the bitcoin-dev mailing list
26 messages, 7 Nov 2023 to 4 Jan 2024. Started by Bryan Bishop.
Peter Todd (3), Ava Chow (2), Ademan (2), Overthefalls (2), Bryan Bishop, Andreas Schildbach, Christopher Allen, David A. Harding, +13 more
Hello,
We would like to request community feedback and proposals on the future of the mailing list.
Our current mailing list host, Linux Foundation, has indicated for years that they have wanted to stop hosting mailing lists, which would mean the bitcoin-dev mailing list would need to move somewhere else. We temporarily avoided that, but recently LF has informed a moderator that they will cease hosting any mailing lists later this year.
In this email, we will go over some of the history, options, and invite discussion ahead of the cutoff. We have some ideas but want to solicit feedback and proposals.
Background ==========
The bitcoin-dev mailing list was originally hosted on Sourceforge.net. The bitcoin development mailing list has been a source of proposals, analysis, and developer discussion for many years in the bitcoin community, with many thousands of participants. Later, the mailing list was migrated to the Linux Foundation, and after that OSUOSL began to help.
Linux Foundation first asked us to move the mailing list in 2017. They internally attempted to migrate all LF mailing lists from mailman2 to mailman3, but ultimately gave up. There were reports of scalability issues with mailman3 for large email communities. Ours definitely qualifies as.. large.
2019 migration plan: LF was to turn off mailman and all lists would migrate to the paid service provider groups.io. Back then we were given accounts to try the groups.io interface and administration features. Apparently we were not the only dev community who resisted change. To our surprise LF gave us several years of reprieve by instead handing the subdomain and server-side data to the non-profit OSUOSL lab who instead operated mailman2 for the past ~4 years.
OSUOSL has for decades been well known for providing server infrastructure for Linux and Open Source development so they were a good fit. This however became an added maintenance burden for the small non-profit with limited resources. Several members of the Bitcoin dev community contributed funding to the lab in support of their Open Source development infrastructure goals. But throwing money at the problem isn’t going to fix the ongoing maintenance burden created by antiquated limitations of mailman2.
Permalinks ==========
Linux Foundation has either offered or agreed to maintain archive permalinks so that content of historic importance is not lost. Fortunately for us while lists.linuxfoundation.org mailman will go down, they have agreed the read-only pipermail archives will remain online. So all old URLs will continue to remain valid. However, the moderators strongly advise that the community supplements with public-inbox instances to have canonical archive urls that are separate from any particular email software host.
Public-Inbox ============
https://public-inbox.org/README.html
“Public Inbox” decentralized archiving - no matter what mailing list server solution is used, anyone can use git to maintain their own mailing list archive and make it available to read on the web.
Public Inbox is a tool that you can run yourself. You can transform your mbox file and it makes it browsable and viewable online. It commits every post to a git repository. It's kind of like a decentralized mail archiving tool. Anyone can publish the mail archive to any web server they wish.
We should try to have one or more canonical archives that are served using public-inbox. But it doesn't matter if these are lost because anyone else can archive the mailing list in the same way and re-publish the archives.
These git commits can also be stamped using opentimestamps, inserting their hashes into the bitcoin blockchain.
LKML mailing list readers often use public-inbox's web interface, and they use the reply-to headers to populate their mail client and reply to threads of interest. This allows their reply to be properly threaded even if they were not a previous subscriber to that mailing list to receive the headers.
public-inbox makes it so that it doesn't really matter where the list is hosted, as pertaining to reading the mailing list. There is still a disruption if the mailing list goes away, but the archives live on and then people can post elsewhere. The archive gets disconnected from the mailing list host in terms of posting. We could have a few canonical URLs for the hosts, separate from the mailing list server.
mailman problems ================
Over the years we have identified a number of problems with mailman2 especially as it pertains to content moderation. There are presently a handful of different moderators, but mailman2 only has a single password for logging into the email management interface. There are no moderator audit logs to see which user (there is no concept of different users) acted on an email. There is no way to mark an email as being investigated by one or more of the moderators. Sometimes, while investigating the veracity of an email, another moderator would come in and approve a suspect email by accident.
Anti spam has been an issue for the moderators. It's relentless. Without access to the underlying server, it has been difficult to fight spam. There is some support for filters in mailman2 but it's not great.
100% active moderation and approval of every email is unsustainable for volunteer moderators. A system that requires moderation is a heavy burden on the moderators and it slows down overall communication and productivity. There's lots of problems with this. Also, moderators can be blamed when they are merely slow while they are not actually censoring.
Rejection emails can optionally be sent to bitcoin-dev-moderation@lists•ozlabs•org but this is an option that a moderator has to remember to type in each time.
Not to mention numerous bugs and vulnerabilities that have accumulated over the years for relatively unmaintained software. (Not disclosed here)
Requirements and considerations ===============================
Looking towards the future, there are a number of properties that seem to be important for the bitcoin-dev mailing list community. First, it is important that backups of the entire archive should be easy for the public to copy or verify so that the system can be brought up elsewhere if necessary.
Second, there seems to be demand for both an email threading interface (using mailing list software) as well as web-accessible interfaces (such as forum software). There seems to be very few options that cater to both email and web. Often, in forum software, email support is limited to email notifications and there is limited if any support for email user participation.
Third, there should be better support for moderator tools and management of the mailing list. See above for complaints about problems with the mailman2 system.
Burdens of running your own mailing list and email server =========================================================
If you have never operated your own MTA you have no idea how difficult it is to keep secure and functional in the face of numerous challenges to deliverability. Anti-spam filtering is essential to prevent forwarding spam. The moment you forward even a single spam message you run the risk of the server IP address being added to blacklists.
The problem of spam filtering is so bad that most IP addresses are presumed guilty even if they have no prior spam history, such as if their network or subnetwork had spam issues in the past.
Even if you put unlimited time into managing your own email server, other people may not accept your email. Or you make one mistake, and then you get into permanent blacklists and it's hard to remove. The spam problem is so bad that most IPs are automatically on a guilty-until-proven-innocent blacklist.
Often there is nothing you can do to get server IP addresses removed from spam blacklists or from "bad reputation" lists.
Ironically, hashcash-style proof-of-work stamps to prevent spam are an appealing solution but not widely used in this community. Or anywhere.
Infinite rejection or forwarding loops happen. They often need to be detected through vigilance and require manual sysadmin intervention to solve.
Bitcoin's dev lists being hosted alongside other Open Source projects was previously protective. If that mailing list server became blacklisted there were a lot of other people who would notice and complain. If we run a Bitcoin-specific mail server we are on our own. 100% of the administrative burden falls upon our own people. There is also nothing we can do if some unknown admin decides they don't like us.
Options =======
Web forums are an interesting option, but often don't have good email user integration. At most you can usually hope for email notifications and an ability to reply by email. It changes the model of the community from push (email) to pull (logging into a forum to read). RSS feeds can help a little bit.
Many other projects have moved from mailing lists to forums (eg https://discuss.python.org/ – see https://lwn.net/Articles/901744/ ; or https://ethresear.ch/), which seem easier to maintain and moderate, and can have lots of advanced features beyond plaintext, maybe-threading and maybe-HTML-markup.
Who would host the forum? Would there be agreement around which forum software to use or which forum host? What about bitcointalk.org or delvingbitcoin.org? There are many options available. Maybe what we actually want isn’t so much a discussion forum, as an 'arxiv of our own' where anons can post BIP drafts and the like?
Given the problems with mailman2, and the decline of email communities in general, it seems that moving to mailman3 would not be a viable long-term option. This leaves us with Google Groups or groups.io as two remaining options.
groups.io is an interesting option: they are a paid service that implements email communities along with online web forum support. However, their public changelog indicates it has been a few years since their last public change. They might be a smaller company and it is unclear how long they will be around or if this would be the right fit for hosting sometimes contentious bitcoin development discussions...
Google Groups is another interesting option, and comes with different tradeoffs. It's the lowest effort to maintain option, and has both an email interface and web forum interface. Users can choose which mode they want to interact with.
For the Google Groups web interface, you can use it with a non-gmail account, but you must create a Google Account which is free to do. it does not require any personal information to do so. This also allows you to add 2FA. Non-gmail non-google users are able to subscribe and post email from their non-gmail non-google email accounts. Tor seems to work for the web interface.
Will Google shut it down, will they cut us off, will they shut down non-google users? The same problem exists with other third-party hosts.
The moderation capabilities for Google Groups and groups.io seem to be comparable. It seems more likely that Google Groups will be able to handle email delivery issues far better than a small resource-constrained operation like groups.io. ((During feedback for this draft, luke-jr indicates that Google Workspaces has been known to use blacklisted IPs for business email!))
On the other hand, groups.io is a paid service and you get what you pay for... hopefully?
Finally, another option is to do literally nothing. It's less work overall. Users can switch to forums or other websites, or private one-on-one communication. It would remove a point of semi-centralization from the bitcoin ecosystem. It would hasten ossification, but on the other hand it would hasten ossification and this could be a negative too. Moderators would be less of a target.
Unfortunately, by doing nothing, there would be no more widely used group email communication system between bitcoin developers. Developers become less coordinated, mayhem and chaos as people go to different communication platforms, a divided community is more vulnerable, etc. BIP1 and BIP2 would need to be revised for other venues.
The main categories of what to move to are: web forums, mailing lists, and hybrids of those two options. Most everything is either self-hosted or you pay someone else to host it. It's kind of the same problem though. It largely depends on how good is the software and unfortunately running your own MTA that forwards mail is not a good option.
Going forward ===========
We'd like to invite feedback and proposals from the community, and see what options are available. One potential option is a migration to Google Groups, but we're open to ideas at this point. We apologize for any inconvenience this disruption has caused.
Bitcoin-dev mailing list moderation team
Bryan Bishop Ruben Somsen Warren Togami various others.
Thanks for writing this up.
I would prefer that we continue to have a mailing list where email is a functional and first class user interface. So that would be to migrate to groups.io or Google Groups. I think Google Groups is probably the better choice of the two.
Although there are concerns that Google would shut down Google Groups or specifically target a bitcoin-dev group, I think both are unlikely to happen. Both Chromium and Android use Google Groups for their mailing lists, so unless those go somewhere else, I doubt Google would unceremoniously kill Google Groups. As for shutting down a bitcoin-dev group specifically, given that there appears to be several thousand groups whose sole purpose is to distribute spam, I don't think Google is in the habit of shutting down groups.
Regardless of what we do, there's always the risk that someone will shut down or stop maintaining the servers for any discussion medium. Even self hosting requires someone to keep the servers up and do maintenance, and that person (or people) could get bored of it, run out of money, get hit by a bus, etc. We're experiencing that right now with the Linux Foundation, and I don't think fear of that should prevent us from moving to a different third party host.
Andrew Chow
On 11/07/2023 10:37 AM, Bryan Bishop via bitcoin-dev wrote:
Quoted text (284 lines)
> Hello, > > We would like to request community feedback and proposals on the future > of the mailing list. > > Our current mailing list host, Linux Foundation, has indicated for years > that they have wanted to stop hosting mailing lists, which would mean > the bitcoin-dev mailing list would need to move somewhere else. We > temporarily avoided that, but recently LF has informed a moderator that > they will cease hosting any mailing lists later this year. > > In this email, we will go over some of the history, options, and invite > discussion ahead of the cutoff. We have some ideas but want to solicit > feedback and proposals. > > Background > ========== > > The bitcoin-dev mailing list was originally hosted on Sourceforge.net. > The bitcoin development mailing list has been a source of proposals, > analysis, and developer discussion for many years in the bitcoin > community, with many thousands of participants. Later, the mailing list > was migrated to the Linux Foundation, and after that OSUOSL began to help. > > Linux Foundation first asked us to move the mailing list in 2017. They > internally attempted to migrate all LF mailing lists from mailman2 to > mailman3, but ultimately gave up. There were reports of scalability > issues with mailman3 for large email communities. Ours definitely > qualifies as.. large. > > 2019 migration plan: LF was to turn off mailman and all lists would > migrate to the paid service provider groups.io <http://groups.io>. Back > then we were given accounts to try the groups.io <http://groups.io> > interface and administration features. Apparently we were not the only > dev community who resisted change. To our surprise LF gave us several > years of reprieve by instead handing the subdomain and server-side data > to the non-profit OSUOSL lab who instead operated mailman2 for the past > ~4 years. > > OSUOSL has for decades been well known for providing server > infrastructure for Linux and Open Source development so they were a good > fit. This however became an added maintenance burden for the small > non-profit with limited resources. Several members of the Bitcoin dev > community contributed funding to the lab in support of their Open Source > development infrastructure goals. But throwing money at the problem > isn’t going to fix the ongoing maintenance burden created by antiquated > limitations of mailman2. > > Permalinks > ========== > > Linux Foundation has either offered or agreed to maintain archive > permalinks so that content of historic importance is not lost. > Fortunately for us while lists.linuxfoundation.org > <http://lists.linuxfoundation.org> mailman will go down, they have > agreed the read-only pipermail archives will remain online. So all old > URLs will continue to remain valid. However, the moderators strongly > advise that the community supplements with public-inbox instances to > have canonical archive urls that are separate from any particular email > software host. > > Public-Inbox > ============ > > https://public-inbox.org/README.html <https://public-inbox.org/README.html> > > “Public Inbox” decentralized archiving - no matter what mailing list > server solution is used, anyone can use git to maintain their own > mailing list archive and make it available to read on the web. > > Public Inbox is a tool that you can run yourself. You can transform your > mbox file and it makes it browsable and viewable online. It commits > every post to a git repository. It's kind of like a decentralized mail > archiving tool. Anyone can publish the mail archive to any web server > they wish. > > We should try to have one or more canonical archives that are served > using public-inbox. But it doesn't matter if these are lost because > anyone else can archive the mailing list in the same way and re-publish > the archives. > > These git commits can also be stamped using opentimestamps, inserting > their hashes into the bitcoin blockchain. > > LKML mailing list readers often use public-inbox's web interface, and > they use the reply-to headers to populate their mail client and reply to > threads of interest. This allows their reply to be properly threaded > even if they were not a previous subscriber to that mailing list to > receive the headers. > > public-inbox makes it so that it doesn't really matter where the list is > hosted, as pertaining to reading the mailing list. There is still a > disruption if the mailing list goes away, but the archives live on and > then people can post elsewhere. The archive gets disconnected from the > mailing list host in terms of posting. We could have a few canonical > URLs for the hosts, separate from the mailing list server. > > mailman problems > ================ > > Over the years we have identified a number of problems with mailman2 > especially as it pertains to content moderation. There are presently a > handful of different moderators, but mailman2 only has a single password > for logging into the email management interface. There are no moderator > audit logs to see which user (there is no concept of different users) > acted on an email. There is no way to mark an email as being > investigated by one or more of the moderators. Sometimes, while > investigating the veracity of an email, another moderator would come in > and approve a suspect email by accident. > > Anti spam has been an issue for the moderators. It's relentless. Without > access to the underlying server, it has been difficult to fight spam. > There is some support for filters in mailman2 but it's not great. > > 100% active moderation and approval of every email is unsustainable for > volunteer moderators. A system that requires moderation is a heavy > burden on the moderators and it slows down overall communication and > productivity. There's lots of problems with this. Also, moderators can > be blamed when they are merely slow while they are not actually censoring. > > Rejection emails can optionally be sent to > bitcoin-dev-moderation@lists•ozlabs•org > <mailto:bitcoin-dev-moderation@lists.ozlabs.org> but this is an option > that a moderator has to remember to type in each time. > > Not to mention numerous bugs and vulnerabilities that have accumulated > over the years for relatively unmaintained software. (Not disclosed here) > > Requirements and considerations > =============================== > > Looking towards the future, there are a number of properties that seem > to be important for the bitcoin-dev mailing list community. First, it is > important that backups of the entire archive should be easy for the > public to copy or verify so that the system can be brought up elsewhere > if necessary. > > Second, there seems to be demand for both an email threading interface > (using mailing list software) as well as web-accessible interfaces (such > as forum software). There seems to be very few options that cater to > both email and web. Often, in forum software, email support is limited > to email notifications and there is limited if any support for email > user participation. > > Third, there should be better support for moderator tools and management > of the mailing list. See above for complaints about problems with the > mailman2 system. > > Burdens of running your own mailing list and email server > ========================================================= > > If you have never operated your own MTA you have no idea how difficult > it is to keep secure and functional in the face of numerous challenges > to deliverability. Anti-spam filtering is essential to prevent > forwarding spam. The moment you forward even a single spam message you > run the risk of the server IP address being added to blacklists. > > The problem of spam filtering is so bad that most IP addresses are > presumed guilty even if they have no prior spam history, such as if > their network or subnetwork had spam issues in the past. > > Even if you put unlimited time into managing your own email server, > other people may not accept your email. Or you make one mistake, and > then you get into permanent blacklists and it's hard to remove. The spam > problem is so bad that most IPs are automatically on a > guilty-until-proven-innocent blacklist. > > Often there is nothing you can do to get server IP addresses removed > from spam blacklists or from "bad reputation" lists. > > Ironically, hashcash-style proof-of-work stamps to prevent spam are an > appealing solution but not widely used in this community. Or anywhere. > > Infinite rejection or forwarding loops happen. They often need to be > detected through vigilance and require manual sysadmin intervention to > solve. > > Bitcoin's dev lists being hosted alongside other Open Source projects > was previously protective. If that mailing list server became > blacklisted there were a lot of other people who would notice and > complain. If we run a Bitcoin-specific mail server we are on our own. > 100% of the administrative burden falls upon our own people. There is > also nothing we can do if some unknown admin decides they don't like us. > > Options > ======= > > Web forums are an interesting option, but often don't have good email > user integration. At most you can usually hope for email notifications > and an ability to reply by email. It changes the model of the community > from push (email) to pull (logging into a forum to read). RSS feeds can > help a little bit. > > Many other projects have moved from mailing lists to forums (eg > https://discuss.python.org/ <https://discuss.python.org/> – see > https://lwn.net/Articles/901744/ <https://lwn.net/Articles/901744/> ; or > https://ethresear.ch/ <https://ethresear.ch/>), which seem easier to > maintain and moderate, and can have lots of advanced features beyond > plaintext, maybe-threading and maybe-HTML-markup. > > Who would host the forum? Would there be agreement around which forum > software to use or which forum host? What about bitcointalk.org > <http://bitcointalk.org> or delvingbitcoin.org > <http://delvingbitcoin.org>? There are many options available. Maybe > what we actually want isn’t so much a discussion forum, as an 'arxiv of > our own' where anons can post BIP drafts and the like? > > Given the problems with mailman2, and the decline of email communities > in general, it seems that moving to mailman3 would not be a viable > long-term option. This leaves us with Google Groups or groups.io > <http://groups.io> as two remaining options. > > groups.io <http://groups.io> is an interesting option: they are a paid > service that implements email communities along with online web forum > support. However, their public changelog indicates it has been a few > years since their last public change. They might be a smaller company > and it is unclear how long they will be around or if this would be the > right fit for hosting sometimes contentious bitcoin development > discussions... > > Google Groups is another interesting option, and comes with different > tradeoffs. It's the lowest effort to maintain option, and has both an > email interface and web forum interface. Users can choose which mode > they want to interact with. > > For the Google Groups web interface, you can use it with a non-gmail > account, but you must create a Google Account which is free to do. it > does not require any personal information to do so. This also allows you > to add 2FA. Non-gmail non-google users are able to subscribe and post > email from their non-gmail non-google email accounts. Tor seems to work > for the web interface. > > Will Google shut it down, will they cut us off, will they shut down > non-google users? The same problem exists with other third-party hosts. > > The moderation capabilities for Google Groups and groups.io > <http://groups.io> seem to be comparable. It seems more likely that > Google Groups will be able to handle email delivery issues far better > than a small resource-constrained operation like groups.io > <http://groups.io>. ((During feedback for this draft, luke-jr indicates > that Google Workspaces has been known to use blacklisted IPs for > business email!)) > > On the other hand, groups.io <http://groups.io> is a paid service and > you get what you pay for... hopefully? > > Finally, another option is to do literally nothing. It's less work > overall. Users can switch to forums or other websites, or private > one-on-one communication. It would remove a point of semi-centralization > from the bitcoin ecosystem. It would hasten ossification, but on the > other hand it would hasten ossification and this could be a negative > too. Moderators would be less of a target. > > Unfortunately, by doing nothing, there would be no more widely used > group email communication system between bitcoin developers. Developers > become less coordinated, mayhem and chaos as people go to different > communication platforms, a divided community is more vulnerable, etc. > BIP1 and BIP2 would need to be revised for other venues. > > The main categories of what to move to are: web forums, mailing lists, > and hybrids of those two options. Most everything is either self-hosted > or you pay someone else to host it. It's kind of the same problem > though. It largely depends on how good is the software and unfortunately > running your own MTA that forwards mail is not a good option. > > Going forward > =========== > > We'd like to invite feedback and proposals from the community, and see > what options are available. One potential option is a migration to > Google Groups, but we're open to ideas at this point. We apologize for > any inconvenience this disruption has caused. > > > Bitcoin-dev mailing list moderation team > > Bryan Bishop > Ruben Somsen > Warren Togami > various others. > > -- > - Bryan > https://twitter.com/kanzure <https://twitter.com/kanzure>
Hi Bryan,
I don't really want my first (and last?) devlist message to be a fairly off-the-cuff post on this topic, but here we go anyway.
At the risk of sounding like a nostr evangelist (I promise I'm not), I want to suggest nostr as a potential replacement to the mailing list. A decent chunk of software would need to be written, but none of the alternatives seem particularly attractive to me. I particularly dislike the idea of locking into a single siloed forum service like the bitcointalk forums. I realize I may be in the minority of course.
Nostr enables the ML team to outsource all of its biggest burdens, if it chooses:
- mail server blocking is N/A to nostr
- Hosting costs are completely outsourced unless the ML team chooses to host a relay.
- Archives and web portal access can be similarly outsourced because any nostr archive is self-authenticating.
- The ML team can also choose to completely outsource moderation, as nostr is (more or less) permissionless by nature. I understand if there is a "blessed" communication system, the ML team would probably prefer to keep it high quality. To that end there are proposals for proof-of-work, and web of trust based blocklists for nostr which are optional for end users. There are other options such as the "moderated communities" proposal which would provide tighter control.
On the user side, the optional moderation is very attractive, allowing controversial threads to exist and continue, without requiring everyone to see them.
The following do not currently exist (to my knowledge) and would need to be implemented to meet the ML's requirements:
- an email gateway to satisfy the bulk of existing ML subscribers This reintroduces issues with mail server blocking of course. - a long-form oriented nostr client (current plain text clients could be used in the meantime)
That admittedly is quite a lot of work, but the second item can be deferred, and the first is not particularly technically challenging, the complications are all on the administration side.
I expect some reflexive NACKs based on the immaturity of the ecosystem but if we have months to prepare, I believe the core requirements can be solidly satisfied in time, the rest can be developed over time, and I believe the advantages are worth careful consideration.
Cheers, Dan
On Tue, Nov 7, 2023 at 9:38 AM Bryan Bishop via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (277 lines)
> Hello, > > We would like to request community feedback and proposals on the future of > the mailing list. > > Our current mailing list host, Linux Foundation, has indicated for years > that they have wanted to stop hosting mailing lists, which would mean the > bitcoin-dev mailing list would need to move somewhere else. We temporarily > avoided that, but recently LF has informed a moderator that they will cease > hosting any mailing lists later this year. > > In this email, we will go over some of the history, options, and invite > discussion ahead of the cutoff. We have some ideas but want to solicit > feedback and proposals. > > Background > ========== > > The bitcoin-dev mailing list was originally hosted on Sourceforge.net. The > bitcoin development mailing list has been a source of proposals, analysis, > and developer discussion for many years in the bitcoin community, with many > thousands of participants. Later, the mailing list was migrated to the > Linux Foundation, and after that OSUOSL began to help. > > Linux Foundation first asked us to move the mailing list in 2017. They > internally attempted to migrate all LF mailing lists from mailman2 to > mailman3, but ultimately gave up. There were reports of scalability issues > with mailman3 for large email communities. Ours definitely qualifies as.. > large. > > 2019 migration plan: LF was to turn off mailman and all lists would > migrate to the paid service provider groups.io. Back then we were given > accounts to try the groups.io interface and administration features. > Apparently we were not the only dev community who resisted change. To our > surprise LF gave us several years of reprieve by instead handing the > subdomain and server-side data to the non-profit OSUOSL lab who instead > operated mailman2 for the past ~4 years. > > OSUOSL has for decades been well known for providing server infrastructure > for Linux and Open Source development so they were a good fit. This however > became an added maintenance burden for the small non-profit with limited > resources. Several members of the Bitcoin dev community contributed funding > to the lab in support of their Open Source development infrastructure > goals. But throwing money at the problem isn’t going to fix the ongoing > maintenance burden created by antiquated limitations of mailman2. > > Permalinks > ========== > > Linux Foundation has either offered or agreed to maintain archive > permalinks so that content of historic importance is not lost. Fortunately > for us while lists.linuxfoundation.org mailman will go down, they have > agreed the read-only pipermail archives will remain online. So all old URLs > will continue to remain valid. However, the moderators strongly advise that > the community supplements with public-inbox instances to have canonical > archive urls that are separate from any particular email software host. > > Public-Inbox > ============ > > https://public-inbox.org/README.html > > “Public Inbox” decentralized archiving - no matter what mailing list > server solution is used, anyone can use git to maintain their own mailing > list archive and make it available to read on the web. > > Public Inbox is a tool that you can run yourself. You can transform your > mbox file and it makes it browsable and viewable online. It commits every > post to a git repository. It's kind of like a decentralized mail archiving > tool. Anyone can publish the mail archive to any web server they wish. > > We should try to have one or more canonical archives that are served using > public-inbox. But it doesn't matter if these are lost because anyone else > can archive the mailing list in the same way and re-publish the archives. > > These git commits can also be stamped using opentimestamps, inserting > their hashes into the bitcoin blockchain. > > LKML mailing list readers often use public-inbox's web interface, and they > use the reply-to headers to populate their mail client and reply to threads > of interest. This allows their reply to be properly threaded even if they > were not a previous subscriber to that mailing list to receive the headers. > > public-inbox makes it so that it doesn't really matter where the list is > hosted, as pertaining to reading the mailing list. There is still a > disruption if the mailing list goes away, but the archives live on and then > people can post elsewhere. The archive gets disconnected from the mailing > list host in terms of posting. We could have a few canonical URLs for the > hosts, separate from the mailing list server. > > mailman problems > ================ > > Over the years we have identified a number of problems with mailman2 > especially as it pertains to content moderation. There are presently a > handful of different moderators, but mailman2 only has a single password > for logging into the email management interface. There are no moderator > audit logs to see which user (there is no concept of different users) acted > on an email. There is no way to mark an email as being investigated by one > or more of the moderators. Sometimes, while investigating the veracity of > an email, another moderator would come in and approve a suspect email by > accident. > > Anti spam has been an issue for the moderators. It's relentless. Without > access to the underlying server, it has been difficult to fight spam. There > is some support for filters in mailman2 but it's not great. > > 100% active moderation and approval of every email is unsustainable for > volunteer moderators. A system that requires moderation is a heavy burden > on the moderators and it slows down overall communication and productivity. > There's lots of problems with this. Also, moderators can be blamed when > they are merely slow while they are not actually censoring. > > Rejection emails can optionally be sent to > bitcoin-dev-moderation@lists•ozlabs•org but this is an option that a > moderator has to remember to type in each time. > > Not to mention numerous bugs and vulnerabilities that have accumulated > over the years for relatively unmaintained software. (Not disclosed here) > > Requirements and considerations > =============================== > > Looking towards the future, there are a number of properties that seem to > be important for the bitcoin-dev mailing list community. First, it is > important that backups of the entire archive should be easy for the public > to copy or verify so that the system can be brought up elsewhere if > necessary. > > Second, there seems to be demand for both an email threading interface > (using mailing list software) as well as web-accessible interfaces (such as > forum software). There seems to be very few options that cater to both > email and web. Often, in forum software, email support is limited to email > notifications and there is limited if any support for email user > participation. > > Third, there should be better support for moderator tools and management > of the mailing list. See above for complaints about problems with the > mailman2 system. > > Burdens of running your own mailing list and email server > ========================================================= > > If you have never operated your own MTA you have no idea how difficult it > is to keep secure and functional in the face of numerous challenges to > deliverability. Anti-spam filtering is essential to prevent forwarding > spam. The moment you forward even a single spam message you run the risk of > the server IP address being added to blacklists. > > The problem of spam filtering is so bad that most IP addresses are > presumed guilty even if they have no prior spam history, such as if their > network or subnetwork had spam issues in the past. > > Even if you put unlimited time into managing your own email server, other > people may not accept your email. Or you make one mistake, and then you get > into permanent blacklists and it's hard to remove. The spam problem is so > bad that most IPs are automatically on a guilty-until-proven-innocent > blacklist. > > Often there is nothing you can do to get server IP addresses removed from > spam blacklists or from "bad reputation" lists. > > Ironically, hashcash-style proof-of-work stamps to prevent spam are an > appealing solution but not widely used in this community. Or anywhere. > > Infinite rejection or forwarding loops happen. They often need to be > detected through vigilance and require manual sysadmin intervention to > solve. > > Bitcoin's dev lists being hosted alongside other Open Source projects was > previously protective. If that mailing list server became blacklisted there > were a lot of other people who would notice and complain. If we run a > Bitcoin-specific mail server we are on our own. 100% of the administrative > burden falls upon our own people. There is also nothing we can do if some > unknown admin decides they don't like us. > > Options > ======= > > Web forums are an interesting option, but often don't have good email user > integration. At most you can usually hope for email notifications and an > ability to reply by email. It changes the model of the community from push > (email) to pull (logging into a forum to read). RSS feeds can help a little > bit. > > Many other projects have moved from mailing lists to forums (eg > https://discuss.python.org/ – see https://lwn.net/Articles/901744/ ; or > https://ethresear.ch/), which seem easier to maintain and moderate, and > can have lots of advanced features beyond plaintext, maybe-threading and > maybe-HTML-markup. > > Who would host the forum? Would there be agreement around which forum > software to use or which forum host? What about bitcointalk.org or > delvingbitcoin.org? There are many options available. Maybe what we > actually want isn’t so much a discussion forum, as an 'arxiv of our own' > where anons can post BIP drafts and the like? > > Given the problems with mailman2, and the decline of email communities in > general, it seems that moving to mailman3 would not be a viable long-term > option. This leaves us with Google Groups or groups.io as two remaining > options. > > groups.io is an interesting option: they are a paid service that > implements email communities along with online web forum support. However, > their public changelog indicates it has been a few years since their last > public change. They might be a smaller company and it is unclear how long > they will be around or if this would be the right fit for hosting sometimes > contentious bitcoin development discussions... > > Google Groups is another interesting option, and comes with different > tradeoffs. It's the lowest effort to maintain option, and has both an email > interface and web forum interface. Users can choose which mode they want to > interact with. > > For the Google Groups web interface, you can use it with a non-gmail > account, but you must create a Google Account which is free to do. it does > not require any personal information to do so. This also allows you to add > 2FA. Non-gmail non-google users are able to subscribe and post email from > their non-gmail non-google email accounts. Tor seems to work for the web > interface. > > Will Google shut it down, will they cut us off, will they shut down > non-google users? The same problem exists with other third-party hosts. > > The moderation capabilities for Google Groups and groups.io seem to be > comparable. It seems more likely that Google Groups will be able to handle > email delivery issues far better than a small resource-constrained > operation like groups.io. ((During feedback for this draft, luke-jr > indicates that Google Workspaces has been known to use blacklisted IPs for > business email!)) > > On the other hand, groups.io is a paid service and you get what you pay > for... hopefully? > > Finally, another option is to do literally nothing. It's less work > overall. Users can switch to forums or other websites, or private > one-on-one communication. It would remove a point of semi-centralization > from the bitcoin ecosystem. It would hasten ossification, but on the other > hand it would hasten ossification and this could be a negative too. > Moderators would be less of a target. > > Unfortunately, by doing nothing, there would be no more widely used group > email communication system between bitcoin developers. Developers become > less coordinated, mayhem and chaos as people go to different communication > platforms, a divided community is more vulnerable, etc. BIP1 and BIP2 would > need to be revised for other venues. > > The main categories of what to move to are: web forums, mailing lists, and > hybrids of those two options. Most everything is either self-hosted or you > pay someone else to host it. It's kind of the same problem though. It > largely depends on how good is the software and unfortunately running your > own MTA that forwards mail is not a good option. > > Going forward > =========== > > We'd like to invite feedback and proposals from the community, and see > what options are available. One potential option is a migration to Google > Groups, but we're open to ideas at this point. We apologize for any > inconvenience this disruption has caused. > > > Bitcoin-dev mailing list moderation team > > Bryan Bishop > Ruben Somsen > Warren Togami > various others. > > -- > - Bryan > https://twitter.com/kanzure > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On 07/11/2023 16.37, Bryan Bishop via bitcoin-dev wrote:
Quoted text (4 lines)
> We would like to request community feedback and proposals on the future > of the mailing list. > > [...]
Have you considered switching to Matrix? It's federated, much like e-mail. It's censorship resistant, in the sense that any participating homeserver keeps a copy of a room. And everyone can run their own homeserver in the Matrix network.
Rooms can be E2E encrypted. For an email list replacement this is probably not relevant, however any person-2-person communication would benefit from being encrypted.
Synapse is the currently best developed home server that implements the Matrix specification. It's much easier to run than an email server + mailing list combo.
Much like E-Mail and IRC, there is a huge ecosystem of Matrix clients to choose from.
I'm aware the UI of most Matrix clients is more suitable for a chat system than lengthy email discussions. However, thanks to "threads", there is no reason it could be used like that.
Hi Dan,
I don't think nostr would be a suitable replacement for the mailing list, although this opinion is biased by the fact that I do not use nostr and find it to be uninteresting.
From my limited understanding of how nostr works, it's not clear to me how a distributed system that uses message broadcast would work in the same way as a mailing list. How would people "subscribe"? How would archives be searched or otherwise be available to people who are not on nostr? How do you distinguish and filter between legitimate dev posts intended for discussion and random crap and shitposting as shows up on social media?
I also don't think that long form text on nostr (or any similar platform) can sufficiently replace email. None of these things seem to contain a way to have a separate subject line as email does. Subjects are immensely important for me as it provides a quick and easy way to filter out things I don't care about reading. I don't want to have read something in before I can decide that I don't care about reading it.
In general, I strongly prefer email, or a platform that has email as a first class user interface, over platforms such as nostr, matrix, or web forums. Email is universal - everyone has one and everyone knows how it works. It dramatically lowers the barrier of entry. Having to make an account somewhere or download some specific client in order to participate will simply result in only the most dedicated participating. Development in open source must be an open process and the barriers to entry should be low.
Lastly, IIRC the plan is to shut down the list by the end of this year. Any solution that requires custom software and bridges to be created are not going to be viable in this time frame.
Andrew
On 11/07/2023 12:03 PM, Ademan via bitcoin-dev wrote:
Quoted text (365 lines)
> Hi Bryan, > > I don't really want my first (and last?) devlist message to be a fairly > off-the-cuff post on this topic, but here we go anyway. > > At the risk of sounding like a nostr evangelist (I promise I'm not), I > want to suggest nostr as a potential replacement to the mailing list. A > decent chunk of software would need to be written, but none of the > alternatives seem particularly attractive to me. I particularly dislike > the idea of locking into a single siloed forum service like the > bitcointalk forums. I realize I may be in the minority of course. > > > Nostr enables the ML team to outsource all of its biggest burdens, if it > chooses: > > - mail server blocking is N/A to nostr > > - Hosting costs are completely outsourced unless the ML team chooses to > host a relay. > > - Archives and web portal access can be similarly outsourced because any > nostr archive is self-authenticating. > > - The ML team can also choose to completely outsource moderation, as > nostr is (more or less) permissionless by nature. > I understand if there is a "blessed" communication system, the ML > team would probably prefer to keep it high quality. To that end there > are proposals for proof-of-work, and web of trust based blocklists for > nostr which are optional for end users. There are other options such as > the "moderated communities" proposal which would provide tighter control. > > > On the user side, the optional moderation is very attractive, allowing > controversial threads to exist and continue, without requiring everyone > to see them. > > > The following do not currently exist (to my knowledge) and would need to > be implemented to meet the ML's requirements: > > - an email gateway to satisfy the bulk of existing ML subscribers > This reintroduces issues with mail server blocking of course. > - a long-form oriented nostr client (current plain text clients could be > used in the meantime) > > That admittedly is quite a lot of work, but the second item can be > deferred, and the first is not particularly technically challenging, the > complications are all on the administration side. > > I expect some reflexive NACKs based on the immaturity of the ecosystem > but if we have months to prepare, I believe the core requirements can be > solidly satisfied in time, the rest can be developed over time, and I > believe the advantages are worth careful consideration. > > Cheers, > Dan > > On Tue, Nov 7, 2023 at 9:38 AM Bryan Bishop via bitcoin-dev > <bitcoin-dev@lists•linuxfoundation•org > <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote: > > Hello, > > We would like to request community feedback and proposals on the > future of the mailing list. > > Our current mailing list host, Linux Foundation, has indicated for > years that they have wanted to stop hosting mailing lists, which > would mean the bitcoin-dev mailing list would need to move somewhere > else. We temporarily avoided that, but recently LF has informed a > moderator that they will cease hosting any mailing lists later this > year. > > In this email, we will go over some of the history, options, and > invite discussion ahead of the cutoff. We have some ideas but want > to solicit feedback and proposals. > > Background > ========== > > The bitcoin-dev mailing list was originally hosted on > Sourceforge.net. The bitcoin development mailing list has been a > source of proposals, analysis, and developer discussion for many > years in the bitcoin community, with many thousands of participants. > Later, the mailing list was migrated to the Linux Foundation, and > after that OSUOSL began to help. > > Linux Foundation first asked us to move the mailing list in 2017. > They internally attempted to migrate all LF mailing lists from > mailman2 to mailman3, but ultimately gave up. There were reports of > scalability issues with mailman3 for large email communities. Ours > definitely qualifies as.. large. > > 2019 migration plan: LF was to turn off mailman and all lists would > migrate to the paid service provider groups.io <http://groups.io>. > Back then we were given accounts to try the groups.io > <http://groups.io> interface and administration features. Apparently > we were not the only dev community who resisted change. To our > surprise LF gave us several years of reprieve by instead handing the > subdomain and server-side data to the non-profit OSUOSL lab who > instead operated mailman2 for the past ~4 years. > > OSUOSL has for decades been well known for providing server > infrastructure for Linux and Open Source development so they were a > good fit. This however became an added maintenance burden for the > small non-profit with limited resources. Several members of the > Bitcoin dev community contributed funding to the lab in support of > their Open Source development infrastructure goals. But throwing > money at the problem isn’t going to fix the ongoing maintenance > burden created by antiquated limitations of mailman2. > > Permalinks > ========== > > Linux Foundation has either offered or agreed to maintain archive > permalinks so that content of historic importance is not lost. > Fortunately for us while lists.linuxfoundation.org > <http://lists.linuxfoundation.org> mailman will go down, they have > agreed the read-only pipermail archives will remain online. So all > old URLs will continue to remain valid. However, the moderators > strongly advise that the community supplements with public-inbox > instances to have canonical archive urls that are separate from any > particular email software host. > > Public-Inbox > ============ > > https://public-inbox.org/README.html > <https://public-inbox.org/README.html> > > “Public Inbox” decentralized archiving - no matter what mailing list > server solution is used, anyone can use git to maintain their own > mailing list archive and make it available to read on the web. > > Public Inbox is a tool that you can run yourself. You can transform > your mbox file and it makes it browsable and viewable online. It > commits every post to a git repository. It's kind of like a > decentralized mail archiving tool. Anyone can publish the mail > archive to any web server they wish. > > We should try to have one or more canonical archives that are served > using public-inbox. But it doesn't matter if these are lost because > anyone else can archive the mailing list in the same way and > re-publish the archives. > > These git commits can also be stamped using opentimestamps, > inserting their hashes into the bitcoin blockchain. > > LKML mailing list readers often use public-inbox's web interface, > and they use the reply-to headers to populate their mail client and > reply to threads of interest. This allows their reply to be properly > threaded even if they were not a previous subscriber to that mailing > list to receive the headers. > > public-inbox makes it so that it doesn't really matter where the > list is hosted, as pertaining to reading the mailing list. There is > still a disruption if the mailing list goes away, but the archives > live on and then people can post elsewhere. The archive gets > disconnected from the mailing list host in terms of posting. We > could have a few canonical URLs for the hosts, separate from the > mailing list server. > > mailman problems > ================ > > Over the years we have identified a number of problems with mailman2 > especially as it pertains to content moderation. There are presently > a handful of different moderators, but mailman2 only has a single > password for logging into the email management interface. There are > no moderator audit logs to see which user (there is no concept of > different users) acted on an email. There is no way to mark an email > as being investigated by one or more of the moderators. Sometimes, > while investigating the veracity of an email, another moderator > would come in and approve a suspect email by accident. > > Anti spam has been an issue for the moderators. It's relentless. > Without access to the underlying server, it has been difficult to > fight spam. There is some support for filters in mailman2 but it's > not great. > > 100% active moderation and approval of every email is unsustainable > for volunteer moderators. A system that requires moderation is a > heavy burden on the moderators and it slows down overall > communication and productivity. There's lots of problems with this. > Also, moderators can be blamed when they are merely slow while they > are not actually censoring. > > Rejection emails can optionally be sent to > bitcoin-dev-moderation@lists•ozlabs•org > <mailto:bitcoin-dev-moderation@lists.ozlabs.org> but this is an > option that a moderator has to remember to type in each time. > > Not to mention numerous bugs and vulnerabilities that have > accumulated over the years for relatively unmaintained software. > (Not disclosed here) > > Requirements and considerations > =============================== > > Looking towards the future, there are a number of properties that > seem to be important for the bitcoin-dev mailing list community. > First, it is important that backups of the entire archive should be > easy for the public to copy or verify so that the system can be > brought up elsewhere if necessary. > > Second, there seems to be demand for both an email threading > interface (using mailing list software) as well as web-accessible > interfaces (such as forum software). There seems to be very few > options that cater to both email and web. Often, in forum software, > email support is limited to email notifications and there is limited > if any support for email user participation. > > Third, there should be better support for moderator tools and > management of the mailing list. See above for complaints about > problems with the mailman2 system. > > Burdens of running your own mailing list and email server > ========================================================= > > If you have never operated your own MTA you have no idea how > difficult it is to keep secure and functional in the face of > numerous challenges to deliverability. Anti-spam filtering is > essential to prevent forwarding spam. The moment you forward even a > single spam message you run the risk of the server IP address being > added to blacklists. > > The problem of spam filtering is so bad that most IP addresses are > presumed guilty even if they have no prior spam history, such as if > their network or subnetwork had spam issues in the past. > > Even if you put unlimited time into managing your own email server, > other people may not accept your email. Or you make one mistake, and > then you get into permanent blacklists and it's hard to remove. The > spam problem is so bad that most IPs are automatically on a > guilty-until-proven-innocent blacklist. > > Often there is nothing you can do to get server IP addresses removed > from spam blacklists or from "bad reputation" lists. > > Ironically, hashcash-style proof-of-work stamps to prevent spam are > an appealing solution but not widely used in this community. Or > anywhere. > > Infinite rejection or forwarding loops happen. They often need to be > detected through vigilance and require manual sysadmin intervention > to solve. > > Bitcoin's dev lists being hosted alongside other Open Source > projects was previously protective. If that mailing list server > became blacklisted there were a lot of other people who would notice > and complain. If we run a Bitcoin-specific mail server we are on our > own. 100% of the administrative burden falls upon our own people. > There is also nothing we can do if some unknown admin decides they > don't like us. > > Options > ======= > > Web forums are an interesting option, but often don't have good > email user integration. At most you can usually hope for email > notifications and an ability to reply by email. It changes the model > of the community from push (email) to pull (logging into a forum to > read). RSS feeds can help a little bit. > > Many other projects have moved from mailing lists to forums (eg > https://discuss.python.org/ <https://discuss.python.org/> – see > https://lwn.net/Articles/901744/ <https://lwn.net/Articles/901744/> > ; or https://ethresear.ch/ <https://ethresear.ch/>), which seem > easier to maintain and moderate, and can have lots of advanced > features beyond plaintext, maybe-threading and maybe-HTML-markup. > > Who would host the forum? Would there be agreement around which > forum software to use or which forum host? What about > bitcointalk.org <http://bitcointalk.org> or delvingbitcoin.org > <http://delvingbitcoin.org>? There are many options available. Maybe > what we actually want isn’t so much a discussion forum, as an 'arxiv > of our own' where anons can post BIP drafts and the like? > > Given the problems with mailman2, and the decline of email > communities in general, it seems that moving to mailman3 would not > be a viable long-term option. This leaves us with Google Groups or > groups.io <http://groups.io> as two remaining options. > > groups.io <http://groups.io> is an interesting option: they are a > paid service that implements email communities along with online web > forum support. However, their public changelog indicates it has been > a few years since their last public change. They might be a smaller > company and it is unclear how long they will be around or if this > would be the right fit for hosting sometimes contentious bitcoin > development discussions... > > Google Groups is another interesting option, and comes with > different tradeoffs. It's the lowest effort to maintain option, and > has both an email interface and web forum interface. Users can > choose which mode they want to interact with. > > For the Google Groups web interface, you can use it with a non-gmail > account, but you must create a Google Account which is free to do. > it does not require any personal information to do so. This also > allows you to add 2FA. Non-gmail non-google users are able to > subscribe and post email from their non-gmail non-google email > accounts. Tor seems to work for the web interface. > > Will Google shut it down, will they cut us off, will they shut down > non-google users? The same problem exists with other third-party hosts. > > The moderation capabilities for Google Groups and groups.io > <http://groups.io> seem to be comparable. It seems more likely that > Google Groups will be able to handle email delivery issues far > better than a small resource-constrained operation like groups.io > <http://groups.io>. ((During feedback for this draft, luke-jr > indicates that Google Workspaces has been known to use blacklisted > IPs for business email!)) > > On the other hand, groups.io <http://groups.io> is a paid service > and you get what you pay for... hopefully? > > Finally, another option is to do literally nothing. It's less work > overall. Users can switch to forums or other websites, or private > one-on-one communication. It would remove a point of > semi-centralization from the bitcoin ecosystem. It would hasten > ossification, but on the other hand it would hasten ossification and > this could be a negative too. Moderators would be less of a target. > > Unfortunately, by doing nothing, there would be no more widely used > group email communication system between bitcoin developers. > Developers become less coordinated, mayhem and chaos as people go to > different communication platforms, a divided community is more > vulnerable, etc. BIP1 and BIP2 would need to be revised for other > venues. > > The main categories of what to move to are: web forums, mailing > lists, and hybrids of those two options. Most everything is either > self-hosted or you pay someone else to host it. It's kind of the > same problem though. It largely depends on how good is the software > and unfortunately running your own MTA that forwards mail is not a > good option. > > Going forward > =========== > > We'd like to invite feedback and proposals from the community, and > see what options are available. One potential option is a migration > to Google Groups, but we're open to ideas at this point. We > apologize for any inconvenience this disruption has caused. > > > Bitcoin-dev mailing list moderation team > > Bryan Bishop > Ruben Somsen > Warren Togami > various others. > > -- > - Bryan > https://twitter.com/kanzure <https://twitter.com/kanzure> > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > <mailto:bitcoin-dev@lists.linuxfoundation.org> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev> >
As Bitcoin-Core already uses GitHub, another possibility is to use the new GitHub discussions feature. We increasingly have been using this at Blockchain Commons as everyone is using already using GitHub. We have also created some GitHub actions to backup discussions so that GitHub will not be a central point of failure -should be possible to create a static page archive using GitHub pages (but have not had budget for that).
For instance, here is the GitHub discussion area for wallet developers working together on Bitcoin wallet interoperability specifications: https://github.com/BlockchainCommons/Gordian-Developer-Community
— Christopher Allen
On 2023-11-07 05:37, Bryan Bishop via bitcoin-dev wrote:
Quoted text (1 line)
> What about [...] delvingbitcoin.org?
I'm only willing to consider discussion groups that provide good archives, so I think it's worth noting that James O'Beirne has written code[1] and is currently maintaining a git repo[2] with a backup of Delving Bitcoin discussion. See his post[3] for additional details.
In addition to providing an archive, I currently find it to be nice way to quickly skim all posts made to the forum since I last checked (plus I see edits)[4]:
$ cd delving-bitcoin-archive/ $ git pull $ git log -p archive/rendered-topics/
I think some technical discussions were already migrating to Delving Bitcoin before the shutdown notice and I expect more discussions to move there in the future even if the current mailing list is relocated to a new platform. Knowing that discussions are archived in a way that I can easily replicate was key to me feeling comfortable putting significant time into reading and writing posts on Delving Bitcoin, so I wanted to share that information here.
-Dave
[1] https://github.com/jamesob/discourse-archive [2] https://github.com/jamesob/delving-bitcoin-archive [3] https://delvingbitcoin.org/t/public-archive-for-delving-bitcoin/87/6 [4] Plus every commit makes me laugh. James O'Beirne's commit robot is called "jamesobot"
Hi Andrew,
Thanks for the thoughtful response. I don't know that you'll find my
responses satisfactory (particularly around moderation), but there are at
least solutions to the objections. Except of course the timeline, which I
got wrong ;-) and means this would be half-baked at best by the time it's
needed.
On Tue, Nov 7, 2023 at 1:06 PM Andrew Chow via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (6 lines)
> Hi Dan, > > I don't think nostr would be a suitable replacement for the mailing > list, although this opinion is biased by the fact that I do not use > nostr and find it to be uninteresting. >
I felt that way for a long time. I still have a number of reservations about it technically, but I'm increasingly impressed, though not for reasons relevant to the discussion.
Quoted text (3 lines)
> From my limited understanding of how nostr works, it's not clear to me > how a distributed system that uses message broadcast would work in the > same way as a mailing list. How would people "subscribe"?
This is already accomplished by existing apps in various ways. There are multiple options, from using labels, or topic tags ( https://github.com/nostr-protocol/nips/tree/master#standardized-tags ) (my preference) or the moderated community approach (see below on moderation)
Quoted text (3 lines)
> How would > archives be searched or otherwise be available to people who are not on > nostr?
Dumb web portals with a familiar interface, existing nostr apps are available as web clients in the interim.
Quoted text (4 lines)
> How do you distinguish and filter between legitimate dev posts > intended for discussion and random crap and shitposting as shows up on > social media? >
I lean strongly towards the topic tag or label approach to constructing the list, which means anyone can post "to the list". Moderation would be applied client side like all nostr clients already do. This is where PoW ( https://github.com/nostr-protocol/nips/blob/master/13.md ) , and/or web-of-trust and labeling ( https://github.com/nostr-protocol/nips/blob/master/32.md ) come in.
Moderation comes "for free" in the moderated communities model ( https://github.com/nostr-protocol/nips/blob/master/72.md ) Note that this approach creates exactly the same moderation burden as the list team already shoulders, and is overall less desirable imho, but it provides the same level of control the current list does.
Quoted text (3 lines)
> I also don't think that long form text on nostr (or any similar > platform) can sufficiently replace email. None of these things seem to > contain a way to have a separate subject line as email does.
"Subject tag" ( https://github.com/nostr-protocol/nips/blob/master/14.md )
Quoted text (9 lines)
> Subjects > are immensely important for me as it provides a quick and easy way to > filter out things I don't care about reading. I don't want to have read > something in before I can decide that I don't care about reading it. > > In general, I strongly prefer email, or a platform that has email as a > first class user interface, over platforms such as nostr, matrix, or web > forums. Email is universal - everyone has one and everyone knows how it > works. It dramatically lowers the barrier of entry.
I think you'd be surprised just how low the barrier to entry in nostr is. You can navigate to the website of any number of nostr apps and click "generate key", but your point about the universality of email is well taken. I think an email bridge would be an essential part of this system. An email bridge would only be less than first-class because email is not as rich as nostr* , from the email-user's perspective it could be no worse than using an existing list server. FWIW I also strongly prefer email over any web forum.
* Unless you want to go wild with non-standard headers
Quoted text (4 lines)
> Having to make an > account somewhere or download some specific client in order to > participate will simply result in only the most dedicated participating. >
I actually think the barrier to entry is exceedingly low, which is part of why we have concern about the spam problem.
Quoted text (7 lines)
> Development in open source must be an open process and the barriers to > entry should be low. > > Lastly, IIRC the plan is to shut down the list by the end of this year. > Any solution that requires custom software and bridges to be created are > not going to be viable in this time frame. >
I have to admit I misread the OP as shutdown at the end of 2024, not 2023, but you're right, barely 2 months is a very tight schedule. Truly addressing the needs of the list in that time frame is unlikely to be possible. FWIW a workable "close enough" is basically possible today, but that is probably unsatisfying.
Certainly if I take the lead, 2024-01-01 is an impossible timeline (especially given my other obligations), but if there is even moderate interest (part of why I bothered to share my ramblings), I could start working on an MVP of the bridge, and a web client for evaluation down the road.
Cheers, Dan
Quoted text (376 lines)
> > Andrew > > On 11/07/2023 12:03 PM, Ademan via bitcoin-dev wrote: > > Hi Bryan, > > > > I don't really want my first (and last?) devlist message to be a fairly > > off-the-cuff post on this topic, but here we go anyway. > > > > At the risk of sounding like a nostr evangelist (I promise I'm not), I > > want to suggest nostr as a potential replacement to the mailing list. A > > decent chunk of software would need to be written, but none of the > > alternatives seem particularly attractive to me. I particularly dislike > > the idea of locking into a single siloed forum service like the > > bitcointalk forums. I realize I may be in the minority of course. > > > > > > Nostr enables the ML team to outsource all of its biggest burdens, if it > > chooses: > > > > - mail server blocking is N/A to nostr > > > > - Hosting costs are completely outsourced unless the ML team chooses to > > host a relay. > > > > - Archives and web portal access can be similarly outsourced because any > > nostr archive is self-authenticating. > > > > - The ML team can also choose to completely outsource moderation, as > > nostr is (more or less) permissionless by nature. > > I understand if there is a "blessed" communication system, the ML > > team would probably prefer to keep it high quality. To that end there > > are proposals for proof-of-work, and web of trust based blocklists for > > nostr which are optional for end users. There are other options such as > > the "moderated communities" proposal which would provide tighter control. > > > > > > On the user side, the optional moderation is very attractive, allowing > > controversial threads to exist and continue, without requiring everyone > > to see them. > > > > > > The following do not currently exist (to my knowledge) and would need to > > be implemented to meet the ML's requirements: > > > > - an email gateway to satisfy the bulk of existing ML subscribers > > This reintroduces issues with mail server blocking of course. > > - a long-form oriented nostr client (current plain text clients could be > > used in the meantime) > > > > That admittedly is quite a lot of work, but the second item can be > > deferred, and the first is not particularly technically challenging, the > > complications are all on the administration side. > > > > I expect some reflexive NACKs based on the immaturity of the ecosystem > > but if we have months to prepare, I believe the core requirements can be > > solidly satisfied in time, the rest can be developed over time, and I > > believe the advantages are worth careful consideration. > > > > Cheers, > > Dan > > > > On Tue, Nov 7, 2023 at 9:38 AM Bryan Bishop via bitcoin-dev > > <bitcoin-dev@lists•linuxfoundation•org > > <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote: > > > > Hello, > > > > We would like to request community feedback and proposals on the > > future of the mailing list. > > > > Our current mailing list host, Linux Foundation, has indicated for > > years that they have wanted to stop hosting mailing lists, which > > would mean the bitcoin-dev mailing list would need to move somewhere > > else. We temporarily avoided that, but recently LF has informed a > > moderator that they will cease hosting any mailing lists later this > > year. > > > > In this email, we will go over some of the history, options, and > > invite discussion ahead of the cutoff. We have some ideas but want > > to solicit feedback and proposals. > > > > Background > > ========== > > > > The bitcoin-dev mailing list was originally hosted on > > Sourceforge.net. The bitcoin development mailing list has been a > > source of proposals, analysis, and developer discussion for many > > years in the bitcoin community, with many thousands of participants. > > Later, the mailing list was migrated to the Linux Foundation, and > > after that OSUOSL began to help. > > > > Linux Foundation first asked us to move the mailing list in 2017. > > They internally attempted to migrate all LF mailing lists from > > mailman2 to mailman3, but ultimately gave up. There were reports of > > scalability issues with mailman3 for large email communities. Ours > > definitely qualifies as.. large. > > > > 2019 migration plan: LF was to turn off mailman and all lists would > > migrate to the paid service provider groups.io <http://groups.io>. > > Back then we were given accounts to try the groups.io > > <http://groups.io> interface and administration features. Apparently > > we were not the only dev community who resisted change. To our > > surprise LF gave us several years of reprieve by instead handing the > > subdomain and server-side data to the non-profit OSUOSL lab who > > instead operated mailman2 for the past ~4 years. > > > > OSUOSL has for decades been well known for providing server > > infrastructure for Linux and Open Source development so they were a > > good fit. This however became an added maintenance burden for the > > small non-profit with limited resources. Several members of the > > Bitcoin dev community contributed funding to the lab in support of > > their Open Source development infrastructure goals. But throwing > > money at the problem isn’t going to fix the ongoing maintenance > > burden created by antiquated limitations of mailman2. > > > > Permalinks > > ========== > > > > Linux Foundation has either offered or agreed to maintain archive > > permalinks so that content of historic importance is not lost. > > Fortunately for us while lists.linuxfoundation.org > > <http://lists.linuxfoundation.org> mailman will go down, they have > > agreed the read-only pipermail archives will remain online. So all > > old URLs will continue to remain valid. However, the moderators > > strongly advise that the community supplements with public-inbox > > instances to have canonical archive urls that are separate from any > > particular email software host. > > > > Public-Inbox > > ============ > > > > https://public-inbox.org/README.html > > <https://public-inbox.org/README.html> > > > > “Public Inbox” decentralized archiving - no matter what mailing list > > server solution is used, anyone can use git to maintain their own > > mailing list archive and make it available to read on the web. > > > > Public Inbox is a tool that you can run yourself. You can transform > > your mbox file and it makes it browsable and viewable online. It > > commits every post to a git repository. It's kind of like a > > decentralized mail archiving tool. Anyone can publish the mail > > archive to any web server they wish. > > > > We should try to have one or more canonical archives that are served > > using public-inbox. But it doesn't matter if these are lost because > > anyone else can archive the mailing list in the same way and > > re-publish the archives. > > > > These git commits can also be stamped using opentimestamps, > > inserting their hashes into the bitcoin blockchain. > > > > LKML mailing list readers often use public-inbox's web interface, > > and they use the reply-to headers to populate their mail client and > > reply to threads of interest. This allows their reply to be properly > > threaded even if they were not a previous subscriber to that mailing > > list to receive the headers. > > > > public-inbox makes it so that it doesn't really matter where the > > list is hosted, as pertaining to reading the mailing list. There is > > still a disruption if the mailing list goes away, but the archives > > live on and then people can post elsewhere. The archive gets > > disconnected from the mailing list host in terms of posting. We > > could have a few canonical URLs for the hosts, separate from the > > mailing list server. > > > > mailman problems > > ================ > > > > Over the years we have identified a number of problems with mailman2 > > especially as it pertains to content moderation. There are presently > > a handful of different moderators, but mailman2 only has a single > > password for logging into the email management interface. There are > > no moderator audit logs to see which user (there is no concept of > > different users) acted on an email. There is no way to mark an email > > as being investigated by one or more of the moderators. Sometimes, > > while investigating the veracity of an email, another moderator > > would come in and approve a suspect email by accident. > > > > Anti spam has been an issue for the moderators. It's relentless. > > Without access to the underlying server, it has been difficult to > > fight spam. There is some support for filters in mailman2 but it's > > not great. > > > > 100% active moderation and approval of every email is unsustainable > > for volunteer moderators. A system that requires moderation is a > > heavy burden on the moderators and it slows down overall > > communication and productivity. There's lots of problems with this. > > Also, moderators can be blamed when they are merely slow while they > > are not actually censoring. > > > > Rejection emails can optionally be sent to > > bitcoin-dev-moderation@lists•ozlabs•org > > <mailto:bitcoin-dev-moderation@lists.ozlabs.org> but this is an > > option that a moderator has to remember to type in each time. > > > > Not to mention numerous bugs and vulnerabilities that have > > accumulated over the years for relatively unmaintained software. > > (Not disclosed here) > > > > Requirements and considerations > > =============================== > > > > Looking towards the future, there are a number of properties that > > seem to be important for the bitcoin-dev mailing list community. > > First, it is important that backups of the entire archive should be > > easy for the public to copy or verify so that the system can be > > brought up elsewhere if necessary. > > > > Second, there seems to be demand for both an email threading > > interface (using mailing list software) as well as web-accessible > > interfaces (such as forum software). There seems to be very few > > options that cater to both email and web. Often, in forum software, > > email support is limited to email notifications and there is limited > > if any support for email user participation. > > > > Third, there should be better support for moderator tools and > > management of the mailing list. See above for complaints about > > problems with the mailman2 system. > > > > Burdens of running your own mailing list and email server > > ========================================================= > > > > If you have never operated your own MTA you have no idea how > > difficult it is to keep secure and functional in the face of > > numerous challenges to deliverability. Anti-spam filtering is > > essential to prevent forwarding spam. The moment you forward even a > > single spam message you run the risk of the server IP address being > > added to blacklists. > > > > The problem of spam filtering is so bad that most IP addresses are > > presumed guilty even if they have no prior spam history, such as if > > their network or subnetwork had spam issues in the past. > > > > Even if you put unlimited time into managing your own email server, > > other people may not accept your email. Or you make one mistake, and > > then you get into permanent blacklists and it's hard to remove. The > > spam problem is so bad that most IPs are automatically on a > > guilty-until-proven-innocent blacklist. > > > > Often there is nothing you can do to get server IP addresses removed > > from spam blacklists or from "bad reputation" lists. > > > > Ironically, hashcash-style proof-of-work stamps to prevent spam are > > an appealing solution but not widely used in this community. Or > > anywhere. > > > > Infinite rejection or forwarding loops happen. They often need to be > > detected through vigilance and require manual sysadmin intervention > > to solve. > > > > Bitcoin's dev lists being hosted alongside other Open Source > > projects was previously protective. If that mailing list server > > became blacklisted there were a lot of other people who would notice > > and complain. If we run a Bitcoin-specific mail server we are on our > > own. 100% of the administrative burden falls upon our own people. > > There is also nothing we can do if some unknown admin decides they > > don't like us. > > > > Options > > ======= > > > > Web forums are an interesting option, but often don't have good > > email user integration. At most you can usually hope for email > > notifications and an ability to reply by email. It changes the model > > of the community from push (email) to pull (logging into a forum to > > read). RSS feeds can help a little bit. > > > > Many other projects have moved from mailing lists to forums (eg > > https://discuss.python.org/ <https://discuss.python.org/> – see > > https://lwn.net/Articles/901744/ <https://lwn.net/Articles/901744/> > > ; or https://ethresear.ch/ <https://ethresear.ch/>), which seem > > easier to maintain and moderate, and can have lots of advanced > > features beyond plaintext, maybe-threading and maybe-HTML-markup. > > > > Who would host the forum? Would there be agreement around which > > forum software to use or which forum host? What about > > bitcointalk.org <http://bitcointalk.org> or delvingbitcoin.org > > <http://delvingbitcoin.org>? There are many options available. Maybe > > what we actually want isn’t so much a discussion forum, as an 'arxiv > > of our own' where anons can post BIP drafts and the like? > > > > Given the problems with mailman2, and the decline of email > > communities in general, it seems that moving to mailman3 would not > > be a viable long-term option. This leaves us with Google Groups or > > groups.io <http://groups.io> as two remaining options. > > > > groups.io <http://groups.io> is an interesting option: they are a > > paid service that implements email communities along with online web > > forum support. However, their public changelog indicates it has been > > a few years since their last public change. They might be a smaller > > company and it is unclear how long they will be around or if this > > would be the right fit for hosting sometimes contentious bitcoin > > development discussions... > > > > Google Groups is another interesting option, and comes with > > different tradeoffs. It's the lowest effort to maintain option, and > > has both an email interface and web forum interface. Users can > > choose which mode they want to interact with. > > > > For the Google Groups web interface, you can use it with a non-gmail > > account, but you must create a Google Account which is free to do. > > it does not require any personal information to do so. This also > > allows you to add 2FA. Non-gmail non-google users are able to > > subscribe and post email from their non-gmail non-google email > > accounts. Tor seems to work for the web interface. > > > > Will Google shut it down, will they cut us off, will they shut down > > non-google users? The same problem exists with other third-party > hosts. > > > > The moderation capabilities for Google Groups and groups.io > > <http://groups.io> seem to be comparable. It seems more likely that > > Google Groups will be able to handle email delivery issues far > > better than a small resource-constrained operation like groups.io > > <http://groups.io>. ((During feedback for this draft, luke-jr > > indicates that Google Workspaces has been known to use blacklisted > > IPs for business email!)) > > > > On the other hand, groups.io <http://groups.io> is a paid service > > and you get what you pay for... hopefully? > > > > Finally, another option is to do literally nothing. It's less work > > overall. Users can switch to forums or other websites, or private > > one-on-one communication. It would remove a point of > > semi-centralization from the bitcoin ecosystem. It would hasten > > ossification, but on the other hand it would hasten ossification and > > this could be a negative too. Moderators would be less of a target. > > > > Unfortunately, by doing nothing, there would be no more widely used > > group email communication system between bitcoin developers. > > Developers become less coordinated, mayhem and chaos as people go to > > different communication platforms, a divided community is more > > vulnerable, etc. BIP1 and BIP2 would need to be revised for other > > venues. > > > > The main categories of what to move to are: web forums, mailing > > lists, and hybrids of those two options. Most everything is either > > self-hosted or you pay someone else to host it. It's kind of the > > same problem though. It largely depends on how good is the software > > and unfortunately running your own MTA that forwards mail is not a > > good option. > > > > Going forward > > =========== > > > > We'd like to invite feedback and proposals from the community, and > > see what options are available. One potential option is a migration > > to Google Groups, but we're open to ideas at this point. We > > apologize for any inconvenience this disruption has caused. > > > > > > Bitcoin-dev mailing list moderation team > > > > Bryan Bishop > > Ruben Somsen > > Warren Togami > > various others. > > > > -- > > - Bryan > > https://twitter.com/kanzure <https://twitter.com/kanzure> > > _______________________________________________ > > bitcoin-dev mailing list > > bitcoin-dev@lists•linuxfoundation•org > > <mailto:bitcoin-dev@lists.linuxfoundation.org> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev> > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
Hi, I also have faced this same problem, and here’s my solution to it:
Use the latest version of https://www.simplemachines.org/ .
This is the same forum software that powered Bitcointalk, Silk Road, etc.
It has many advantages over every other platform out there:
1. It has great anti-spam prevention tools. 2. It gives you the tools you’ve requested with respect to moderation (individual moderation accounts with customizable permissions). 3. It is simple to use. 4. It is pretty and well designed, and allows you to organize threads and forums really well (unlike Discourse). 5. It doesn’t make unnecessary use of JavaScript (unlike Discourse), and therefore works well with all search engines (present and future). 6. You can self-host it, and migrate it to another server if needed, allowing the community to maintain full control over the data (unlike Microsoft/Github). 7. It supports great search functionality. 8. It is open source, and has many community plugins. 9. It runs anywhere PHP runs. 10. It is fast. 11. It has a well established history of powering Bitcoin communities. It has been with us from Day 1.
After using phpBB for years, researching other forums software, I switched to SMF, and stuck with it. I’m happy I did so. I recommend it.
Kind regards, Greg Slepak
Quoted text (3 lines)
> On Nov 7, 2023, at 7:37 AM, Bryan Bishop via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote: > > [moving]
I also think that good archives are extremely important. Far more important than being a medium of discussion is capturing all of that discussion for posterity. An unbelievable amount of knowledge capital has been built up in the mailing list over the years and given that Bitcoin is a system that needs to survive complete turnover in its contributor base, it's of extreme importance that we have a system that can capture the archive.
While Nostr might be good towards the end of being very resilient it isn't mature enough to have good UX's built up around it wherein people with a variety of backgrounds can engage it. Personally, I think the email UX leaves a lot to be desired but at least it's accessible to a lot of people. I don't think I can say the same for Nostr yet.
I won't opine much further on the solution but I think the properties we need to solve for are:
1. Archive is effectively permanent 2. Accessible to a wide audience 3. Data format is not proprietary and isn't tied to the success or failure of a particular organization
In principle I think that Nostr can offer a lot in the long term towards this goal, but it isn't really an immediate solution to this problem.
Keags
On Tue, Nov 7, 2023 at 12:07 PM David A. Harding via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (36 lines)
> On 2023-11-07 05:37, Bryan Bishop via bitcoin-dev wrote: > > What about [...] delvingbitcoin.org? > > I'm only willing to consider discussion groups that provide good > archives, so I think it's worth noting that James O'Beirne has written > code[1] and is currently maintaining a git repo[2] with a backup of > Delving Bitcoin discussion. See his post[3] for additional details. > > In addition to providing an archive, I currently find it to be nice way > to quickly skim all posts made to the forum since I last checked (plus I > see edits)[4]: > > $ cd delving-bitcoin-archive/ > $ git pull > $ git log -p archive/rendered-topics/ > > I think some technical discussions were already migrating to Delving > Bitcoin before the shutdown notice and I expect more discussions to move > there in the future even if the current mailing list is relocated to a > new platform. Knowing that discussions are archived in a way that I can > easily replicate was key to me feeling comfortable putting significant > time into reading and writing posts on Delving Bitcoin, so I wanted to > share that information here. > > -Dave > > [1] https://github.com/jamesob/discourse-archive > [2] https://github.com/jamesob/delving-bitcoin-archive > [3] https://delvingbitcoin.org/t/public-archive-for-delving-bitcoin/87/6 > [4] Plus every commit makes me laugh. James O'Beirne's commit robot is > called "jamesobot" > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
Hi,
This impellent deadline could be took with enthusiasm from people that are anxious to experiment with new protocols and platforms that can replicate mailing lists and offer, in theory, better solutions. I think this enthusiasm is totally positive and I encourage them to work on that ideas.
But I also think that this mailing list fills a very particoular need of communication in the bitcoin space. The stream of ideas hosted here is strictly dependant on the form it assumes when formalized in the peculiar format of mails-threads. Migrating these technical discussions to a forum or a pseudo-group-chat wouldn't replace this mailing list, even if the moderators behind and most of the participants would be the same. It would eventually be a new and unstable solution, with no-guarantee to preserve the same goals reached here.
Today exist a lot of places where people can exchange ideas about bitcoin; if new platforms will emerge as better suited to hosts BIP drafts and technical discussions, people will move organically through them. In my opinion, "finding a new platform" is only marginally correlated to our main topic here.
If our problem is helping decide the "future of bitcoin-dev mailing list", the only two solutions to me appear to tautologically be:
1) Give continuity to bitcoin-dev mailing list with a ready drop-in replacement.
2) Don't give continuity the bitcoin-dev mailing list.
In the case 1) a solution could be find a new host for the mailing list and work around the problems exposed.
In the case 2) is possible to do nothing OR to propose a new solution as a sort of "spiritual continuation" of bitcoin-dev mailing list, and eventually see if people will converge on it.
Understanding all the difficulty behind the management of the bitcoin-dev mailing list, I think it has worked very well for many years, and I hope it will work for the years to come. I also want to say thanks to all the people behind this mailing list for all your work and effort.
---PM
Il 7 novembre 2023 16:37:22 CET, Bryan Bishop via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> ha scritto:
Quoted text (272 lines)
>Hello, > >We would like to request community feedback and proposals on the future of >the mailing list. > >Our current mailing list host, Linux Foundation, has indicated for years >that they have wanted to stop hosting mailing lists, which would mean the >bitcoin-dev mailing list would need to move somewhere else. We temporarily >avoided that, but recently LF has informed a moderator that they will cease >hosting any mailing lists later this year. > >In this email, we will go over some of the history, options, and invite >discussion ahead of the cutoff. We have some ideas but want to solicit >feedback and proposals. > >Background >========== > >The bitcoin-dev mailing list was originally hosted on Sourceforge.net. The >bitcoin development mailing list has been a source of proposals, analysis, >and developer discussion for many years in the bitcoin community, with many >thousands of participants. Later, the mailing list was migrated to the >Linux Foundation, and after that OSUOSL began to help. > >Linux Foundation first asked us to move the mailing list in 2017. They >internally attempted to migrate all LF mailing lists from mailman2 to >mailman3, but ultimately gave up. There were reports of scalability issues >with mailman3 for large email communities. Ours definitely qualifies as.. >large. > >2019 migration plan: LF was to turn off mailman and all lists would migrate >to the paid service provider groups.io. Back then we were given accounts to >try the groups.io interface and administration features. Apparently we were >not the only dev community who resisted change. To our surprise LF gave us >several years of reprieve by instead handing the subdomain and server-side >data to the non-profit OSUOSL lab who instead operated mailman2 for the >past ~4 years. > >OSUOSL has for decades been well known for providing server infrastructure >for Linux and Open Source development so they were a good fit. This however >became an added maintenance burden for the small non-profit with limited >resources. Several members of the Bitcoin dev community contributed funding >to the lab in support of their Open Source development infrastructure >goals. But throwing money at the problem isn’t going to fix the ongoing >maintenance burden created by antiquated limitations of mailman2. > >Permalinks >========== > >Linux Foundation has either offered or agreed to maintain archive >permalinks so that content of historic importance is not lost. Fortunately >for us while lists.linuxfoundation.org mailman will go down, they have >agreed the read-only pipermail archives will remain online. So all old URLs >will continue to remain valid. However, the moderators strongly advise that >the community supplements with public-inbox instances to have canonical >archive urls that are separate from any particular email software host. > >Public-Inbox >============ > >https://public-inbox.org/README.html > >“Public Inbox” decentralized archiving - no matter what mailing list server >solution is used, anyone can use git to maintain their own mailing list >archive and make it available to read on the web. > >Public Inbox is a tool that you can run yourself. You can transform your >mbox file and it makes it browsable and viewable online. It commits every >post to a git repository. It's kind of like a decentralized mail archiving >tool. Anyone can publish the mail archive to any web server they wish. > >We should try to have one or more canonical archives that are served using >public-inbox. But it doesn't matter if these are lost because anyone else >can archive the mailing list in the same way and re-publish the archives. > >These git commits can also be stamped using opentimestamps, inserting their >hashes into the bitcoin blockchain. > >LKML mailing list readers often use public-inbox's web interface, and they >use the reply-to headers to populate their mail client and reply to threads >of interest. This allows their reply to be properly threaded even if they >were not a previous subscriber to that mailing list to receive the headers. > >public-inbox makes it so that it doesn't really matter where the list is >hosted, as pertaining to reading the mailing list. There is still a >disruption if the mailing list goes away, but the archives live on and then >people can post elsewhere. The archive gets disconnected from the mailing >list host in terms of posting. We could have a few canonical URLs for the >hosts, separate from the mailing list server. > >mailman problems >================ > >Over the years we have identified a number of problems with mailman2 >especially as it pertains to content moderation. There are presently a >handful of different moderators, but mailman2 only has a single password >for logging into the email management interface. There are no moderator >audit logs to see which user (there is no concept of different users) acted >on an email. There is no way to mark an email as being investigated by one >or more of the moderators. Sometimes, while investigating the veracity of >an email, another moderator would come in and approve a suspect email by >accident. > >Anti spam has been an issue for the moderators. It's relentless. Without >access to the underlying server, it has been difficult to fight spam. There >is some support for filters in mailman2 but it's not great. > >100% active moderation and approval of every email is unsustainable for >volunteer moderators. A system that requires moderation is a heavy burden >on the moderators and it slows down overall communication and productivity. >There's lots of problems with this. Also, moderators can be blamed when >they are merely slow while they are not actually censoring. > >Rejection emails can optionally be sent to >bitcoin-dev-moderation@lists•ozlabs•org but this is an option that a >moderator has to remember to type in each time. > >Not to mention numerous bugs and vulnerabilities that have accumulated over >the years for relatively unmaintained software. (Not disclosed here) > >Requirements and considerations >=============================== > >Looking towards the future, there are a number of properties that seem to >be important for the bitcoin-dev mailing list community. First, it is >important that backups of the entire archive should be easy for the public >to copy or verify so that the system can be brought up elsewhere if >necessary. > >Second, there seems to be demand for both an email threading interface >(using mailing list software) as well as web-accessible interfaces (such as >forum software). There seems to be very few options that cater to both >email and web. Often, in forum software, email support is limited to email >notifications and there is limited if any support for email user >participation. > >Third, there should be better support for moderator tools and management of >the mailing list. See above for complaints about problems with the mailman2 >system. > >Burdens of running your own mailing list and email server >========================================================= > >If you have never operated your own MTA you have no idea how difficult it >is to keep secure and functional in the face of numerous challenges to >deliverability. Anti-spam filtering is essential to prevent forwarding >spam. The moment you forward even a single spam message you run the risk of >the server IP address being added to blacklists. > >The problem of spam filtering is so bad that most IP addresses are presumed >guilty even if they have no prior spam history, such as if their network or >subnetwork had spam issues in the past. > >Even if you put unlimited time into managing your own email server, other >people may not accept your email. Or you make one mistake, and then you get >into permanent blacklists and it's hard to remove. The spam problem is so >bad that most IPs are automatically on a guilty-until-proven-innocent >blacklist. > >Often there is nothing you can do to get server IP addresses removed from >spam blacklists or from "bad reputation" lists. > >Ironically, hashcash-style proof-of-work stamps to prevent spam are an >appealing solution but not widely used in this community. Or anywhere. > >Infinite rejection or forwarding loops happen. They often need to be >detected through vigilance and require manual sysadmin intervention to >solve. > >Bitcoin's dev lists being hosted alongside other Open Source projects was >previously protective. If that mailing list server became blacklisted there >were a lot of other people who would notice and complain. If we run a >Bitcoin-specific mail server we are on our own. 100% of the administrative >burden falls upon our own people. There is also nothing we can do if some >unknown admin decides they don't like us. > >Options >======= > >Web forums are an interesting option, but often don't have good email user >integration. At most you can usually hope for email notifications and an >ability to reply by email. It changes the model of the community from push >(email) to pull (logging into a forum to read). RSS feeds can help a little >bit. > >Many other projects have moved from mailing lists to forums (eg >https://discuss.python.org/ – see https://lwn.net/Articles/901744/ ; or >https://ethresear.ch/), which seem easier to maintain and moderate, and can >have lots of advanced features beyond plaintext, maybe-threading and >maybe-HTML-markup. > >Who would host the forum? Would there be agreement around which forum >software to use or which forum host? What about bitcointalk.org or >delvingbitcoin.org? There are many options available. Maybe what we >actually want isn’t so much a discussion forum, as an 'arxiv of our own' >where anons can post BIP drafts and the like? > >Given the problems with mailman2, and the decline of email communities in >general, it seems that moving to mailman3 would not be a viable long-term >option. This leaves us with Google Groups or groups.io as two remaining >options. > >groups.io is an interesting option: they are a paid service that implements >email communities along with online web forum support. However, their >public changelog indicates it has been a few years since their last public >change. They might be a smaller company and it is unclear how long they >will be around or if this would be the right fit for hosting sometimes >contentious bitcoin development discussions... > >Google Groups is another interesting option, and comes with different >tradeoffs. It's the lowest effort to maintain option, and has both an email >interface and web forum interface. Users can choose which mode they want to >interact with. > >For the Google Groups web interface, you can use it with a non-gmail >account, but you must create a Google Account which is free to do. it does >not require any personal information to do so. This also allows you to add >2FA. Non-gmail non-google users are able to subscribe and post email from >their non-gmail non-google email accounts. Tor seems to work for the web >interface. > >Will Google shut it down, will they cut us off, will they shut down >non-google users? The same problem exists with other third-party hosts. > >The moderation capabilities for Google Groups and groups.io seem to be >comparable. It seems more likely that Google Groups will be able to handle >email delivery issues far better than a small resource-constrained >operation like groups.io. ((During feedback for this draft, luke-jr >indicates that Google Workspaces has been known to use blacklisted IPs for >business email!)) > >On the other hand, groups.io is a paid service and you get what you pay >for... hopefully? > >Finally, another option is to do literally nothing. It's less work overall. >Users can switch to forums or other websites, or private one-on-one >communication. It would remove a point of semi-centralization from the >bitcoin ecosystem. It would hasten ossification, but on the other hand it >would hasten ossification and this could be a negative too. Moderators >would be less of a target. > >Unfortunately, by doing nothing, there would be no more widely used group >email communication system between bitcoin developers. Developers become >less coordinated, mayhem and chaos as people go to different communication >platforms, a divided community is more vulnerable, etc. BIP1 and BIP2 would >need to be revised for other venues. > >The main categories of what to move to are: web forums, mailing lists, and >hybrids of those two options. Most everything is either self-hosted or you >pay someone else to host it. It's kind of the same problem though. It >largely depends on how good is the software and unfortunately running your >own MTA that forwards mail is not a good option. > >Going forward >=========== > >We'd like to invite feedback and proposals from the community, and see what >options are available. One potential option is a migration to Google >Groups, but we're open to ideas at this point. We apologize for any >inconvenience this disruption has caused. > > >Bitcoin-dev mailing list moderation team > >Bryan Bishop >Ruben Somsen >Warren Togami >various others. > >-- >- Bryan >https://twitter.com/kanzure
I think GitHub Discussions is a great idea. If we are considering proprietary options like Google Groups, then we should definitely consider Discussions.
1. Guaranteed that nearly everyone participating here already has a GH account. 2. Offers many moderation options. 3. Good formatting abilities(tables!). 4. Can @ people. 5. Ability to categorize, close, lock discussions, etc. 6. Many great potential opportunities for automation via Actions. 7. Comes with added benefits such as a wiki, issues, etc.
My one catch is that I do not know what kind of interactions you can have with Discussions via email. This seems to be an important feature for many. What level of email notifications can you receive? Can you respond via email?
Quoted text (13 lines)
> On Nov 7, 2023, at 3:16 PM, Christopher Allen via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote: > > > As Bitcoin-Core already uses GitHub, another possibility is to use the new GitHub discussions feature. We increasingly have been using this at Blockchain Commons as everyone is using already using GitHub. We have also created some GitHub actions to backup discussions so that GitHub will not be a central point of failure -should be possible to create a static page archive using GitHub pages (but have not had budget for that). > > For instance, here is the GitHub discussion area for wallet developers working together on Bitcoin wallet interoperability specifications: > https://github.com/BlockchainCommons/Gordian-Developer-Community > > — Christopher Allen > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
On Tue, Nov 07, 2023 at 11:41:59AM -0800, Christopher Allen via bitcoin-dev wrote:
Quoted text (10 lines)
> As Bitcoin-Core already uses GitHub, another possibility is to use the new > GitHub discussions feature. We increasingly have been using this at > Blockchain Commons as everyone is using already using GitHub. We have also > created some GitHub actions to backup discussions so that GitHub will not > be a central point of failure -should be possible to create a static page > archive using GitHub pages (but have not had budget for that). > > For instance, here is the GitHub discussion area for wallet developers > working together on Bitcoin wallet interoperability specifications: > https://github.com/BlockchainCommons/Gordian-Developer-Community
Strong NACK.
bitcoin-dev should be independent of Bitcoin Core.
Also, a very useful thing that a mailing list does that GitHub does not is cryptographic signatures, both obvious like PGP, and less obvious like DKIM. We should not be moving even more discussion to mediums where authors aren't properly signing their messages.
The user experience of GitHub and similar web forums is poor too. It's much nicer to be able to reply to messages offline, asyncronously, regardless of whether or not you happen to have a good internet connection at the time.
On Tue, Nov 07, 2023 at 11:03:30AM -0600, Ademan via bitcoin-dev wrote:
Quoted text (11 lines)
> Hi Bryan, > > I don't really want my first (and last?) devlist message to be a fairly > off-the-cuff post on this topic, but here we go anyway. > > At the risk of sounding like a nostr evangelist (I promise I'm not), I want > to suggest nostr as a potential replacement to the mailing list. A decent > chunk of software would need to be written, but none of the alternatives > seem particularly attractive to me. I particularly dislike the idea of > locking into a single siloed forum service like the bitcointalk forums. I > realize I may be in the minority of course.
Strong NACK on nostr. It's a badly designed, centralized, protocol that needs a significant redesign to be usable. While off topic for this mailing list, some of its many issues include:
* Reliance on single-key, cryptography that often results in people having their keys compromised. This is a serious problem in the context of bitcoin-dev, where faked messages published could easily have market-moving results.
* Inability to mirror relays: since nostr deliberately ignores the lessons of blockchains, there is no way to be sure that you have a complete set of messages from a given person, for a given topic, etc.
* Highly centralized design: since mirroring relays isn't reliable, in reality nostr operates in a highly centralized fashion, dependent on a tiny number of relays that can't be easily replaced if taken down.
On Tue, Nov 07, 2023 at 09:37:22AM -0600, Bryan Bishop via bitcoin-dev wrote:
Quoted text (3 lines)
> Anti spam has been an issue for the moderators. It's relentless. Without > access to the underlying server, it has been difficult to fight spam. There > is some support for filters in mailman2 but it's not great.
Since this is a technical mailing list it would be fine to require people to pay a non-refundable anti-spam fee, eg via lightning, to gain the ability to send messages. While this would require some custom software, it's probably even possible to implement this if a third party is used for hosting, provided they have some kind of API.
On Tue, Nov 07, 2023 at 09:37:22AM -0600, Bryan Bishop via bitcoin-dev wrote:
Quoted text (3 lines)
> Web forums are an interesting option, but often don't have good email user > integration. > What about bitcointalk.org or delvingbitcoin.org?
delvingbitcoin.org is something I setup; it's a self-hosted discourse instance. (You don't have to self-host discourse, but not doing so limits the number of admins/moderators, the plugins you can use, and the APIs you can access)
For what it's worth, I think (discourse) forums have significant advantages over email for technical discussion:
* much better markup: you can write LaTeX for doing maths, you can have graphviz or mermaid diagrams generated directly from text, you can do formatting without having to worry about HTML email. because that's done direct from markup, you can also quote such things in replies, or easily create a modified equation/diagram if desired, things that are much harder if equations/diagrams are image/pdf attachments.
* consistent threading/quoting: you don't have to rely on email clients to get threading/quoting correct in order to link replies with the original message
* having topics/replies, rather than everything being an individual email, tends to make it easier to avoid being distracted by followups to a topic you're not interested in.
* you can do reactions (heart / thumbs up / etc) instead of "me too" posts, minimising the impact of low-content responses on readers, without doing away with those responses entirely.
* after the fact moderation: with mailing lists, moderation can only be a choice between "send this post to every subscriber" or not, and the choice obviously has to be made before anyone sees the posts; forums allow off-topic/unconstructive posts to be removed or edited.
Compared to mailing-lists-as-a-service, a self-hosted forum has a few other possible benefits:
* it's easier to setup areas for additional topics, without worrying you're going to be forced into an arbitrarily higher pricing tier
* you can setup spaces for private working groups. (and those groups can make their internal discussions public after the fact, if desired)
* you can use plugin interfaces/APIs to link up with external resources
There are a few disadvantages too:
* discourse isn't lightweight -- you need a whole bunch of infrastructure to go from the markdown posts to the actual rendered posts/comments; so backups of just the markdown text isn't really "complete"
* discourse is quite actively developed -- so it could be possible that posts that use particular features/plugins (eg to generate diagrams) will go stale eventually as the software changes, and stop being rendered correctly
* discourse gathers a moderate amount of non-public/potentially private data (eg email addresses, passwords, IP addresses, login times) that may make backups and admin access sensitive (which is why there's a git archive generated by a bot for delvingbitcoin, rather than raw database dumps)
There are quite a few open source projects using discourse instances, eg:
Python: https://discuss.python.org/ Ruby on Rails: https://discuss.rubyonrails.org/ LLVM: https://discourse.llvm.org/ Jupyter: https://discourse.jupyter.org/ Fedora: https://discussion.fedoraproject.org/ Ubuntu: https://discourse.ubuntu.com/ Haskell: https://discourse.haskell.org/
There's also various crypto projects using it:
Eth research: https://ethresear.ch/ Chia: https://developers.chia.net/
There's a couple of LWN articles on Python's adoption of discourse that I found interesting, fwiw:
https://lwn.net/Articles/901744/ [2022-07-20] https://lwn.net/Articles/674271/ [2016-02-03]
I don't think this needs to be an "either-or" question -- better to have technical discussions about bitcoin in many places and in many formats, rather than just one -- but I thought I'd take the opportunity to write out why I thought discourse was worth spending some time on in this context.
Cheers, aj
On 2023-11-07 17:12, Andrew Chow via bitcoin-dev wrote:
Quoted text (4 lines)
> I would prefer that we continue to have a mailing list where email is a > functional and first class user interface. So that would be to migrate > to groups.io or Google Groups. I think Google Groups is probably the > better choice of the two.
+1 to migrating to a different email service. That seems like the most straightforward solution.
On 2023-11-08 04:56, Anthony Towns via bitcoin-dev wrote:
Quoted text (1 line)
> delvingbitcoin.org is something I setup
Cool, thanks! It's great to have more channels of discussion. Same with IRC etc.
Cheers, -Yancy
Quoted text (335 lines)
> Andrew Chow > > On 11/07/2023 10:37 AM, Bryan Bishop via bitcoin-dev wrote: > >> Hello, >> >> We would like to request community feedback and proposals on the >> future >> of the mailing list. >> >> Our current mailing list host, Linux Foundation, has indicated for >> years >> that they have wanted to stop hosting mailing lists, which would mean >> the bitcoin-dev mailing list would need to move somewhere else. We >> temporarily avoided that, but recently LF has informed a moderator >> that >> they will cease hosting any mailing lists later this year. >> >> In this email, we will go over some of the history, options, and >> invite >> discussion ahead of the cutoff. We have some ideas but want to solicit >> feedback and proposals. >> >> Background >> ========== >> >> The bitcoin-dev mailing list was originally hosted on Sourceforge.net. >> The bitcoin development mailing list has been a source of proposals, >> analysis, and developer discussion for many years in the bitcoin >> community, with many thousands of participants. Later, the mailing >> list >> was migrated to the Linux Foundation, and after that OSUOSL began to >> help. >> >> Linux Foundation first asked us to move the mailing list in 2017. They >> internally attempted to migrate all LF mailing lists from mailman2 to >> mailman3, but ultimately gave up. There were reports of scalability >> issues with mailman3 for large email communities. Ours definitely >> qualifies as.. large. >> >> 2019 migration plan: LF was to turn off mailman and all lists would >> migrate to the paid service provider groups.io <http://groups.io>. >> Back >> then we were given accounts to try the groups.io <http://groups.io> >> interface and administration features. Apparently we were not the only >> dev community who resisted change. To our surprise LF gave us several >> years of reprieve by instead handing the subdomain and server-side >> data >> to the non-profit OSUOSL lab who instead operated mailman2 for the >> past >> ~4 years. >> >> OSUOSL has for decades been well known for providing server >> infrastructure for Linux and Open Source development so they were a >> good >> fit. This however became an added maintenance burden for the small >> non-profit with limited resources. Several members of the Bitcoin dev >> community contributed funding to the lab in support of their Open >> Source >> development infrastructure goals. But throwing money at the problem >> isn't going to fix the ongoing maintenance burden created by >> antiquated >> limitations of mailman2. >> >> Permalinks >> ========== >> >> Linux Foundation has either offered or agreed to maintain archive >> permalinks so that content of historic importance is not lost. >> Fortunately for us while lists.linuxfoundation.org >> <http://lists.linuxfoundation.org> mailman will go down, they have >> agreed the read-only pipermail archives will remain online. So all old >> URLs will continue to remain valid. However, the moderators strongly >> advise that the community supplements with public-inbox instances to >> have canonical archive urls that are separate from any particular >> email >> software host. >> >> Public-Inbox >> ============ >> >> https://public-inbox.org/README.html >> <https://public-inbox.org/README.html> >> >> "Public Inbox" decentralized archiving - no matter what mailing list >> server solution is used, anyone can use git to maintain their own >> mailing list archive and make it available to read on the web. >> >> Public Inbox is a tool that you can run yourself. You can transform >> your >> mbox file and it makes it browsable and viewable online. It commits >> every post to a git repository. It's kind of like a decentralized mail >> archiving tool. Anyone can publish the mail archive to any web server >> they wish. >> >> We should try to have one or more canonical archives that are served >> using public-inbox. But it doesn't matter if these are lost because >> anyone else can archive the mailing list in the same way and >> re-publish >> the archives. >> >> These git commits can also be stamped using opentimestamps, inserting >> their hashes into the bitcoin blockchain. >> >> LKML mailing list readers often use public-inbox's web interface, and >> they use the reply-to headers to populate their mail client and reply >> to >> threads of interest. This allows their reply to be properly threaded >> even if they were not a previous subscriber to that mailing list to >> receive the headers. >> >> public-inbox makes it so that it doesn't really matter where the list >> is >> hosted, as pertaining to reading the mailing list. There is still a >> disruption if the mailing list goes away, but the archives live on and >> then people can post elsewhere. The archive gets disconnected from the >> mailing list host in terms of posting. We could have a few canonical >> URLs for the hosts, separate from the mailing list server. >> >> mailman problems >> ================ >> >> Over the years we have identified a number of problems with mailman2 >> especially as it pertains to content moderation. There are presently a >> handful of different moderators, but mailman2 only has a single >> password >> for logging into the email management interface. There are no >> moderator >> audit logs to see which user (there is no concept of different users) >> acted on an email. There is no way to mark an email as being >> investigated by one or more of the moderators. Sometimes, while >> investigating the veracity of an email, another moderator would come >> in >> and approve a suspect email by accident. >> >> Anti spam has been an issue for the moderators. It's relentless. >> Without >> access to the underlying server, it has been difficult to fight spam. >> There is some support for filters in mailman2 but it's not great. >> >> 100% active moderation and approval of every email is unsustainable >> for >> volunteer moderators. A system that requires moderation is a heavy >> burden on the moderators and it slows down overall communication and >> productivity. There's lots of problems with this. Also, moderators can >> be blamed when they are merely slow while they are not actually >> censoring. >> >> Rejection emails can optionally be sent to >> bitcoin-dev-moderation@lists•ozlabs•org >> <mailto:bitcoin-dev-moderation@lists.ozlabs.org> but this is an option >> that a moderator has to remember to type in each time. >> >> Not to mention numerous bugs and vulnerabilities that have accumulated >> over the years for relatively unmaintained software. (Not disclosed >> here) >> >> Requirements and considerations >> =============================== >> >> Looking towards the future, there are a number of properties that seem >> to be important for the bitcoin-dev mailing list community. First, it >> is >> important that backups of the entire archive should be easy for the >> public to copy or verify so that the system can be brought up >> elsewhere >> if necessary. >> >> Second, there seems to be demand for both an email threading interface >> (using mailing list software) as well as web-accessible interfaces >> (such >> as forum software). There seems to be very few options that cater to >> both email and web. Often, in forum software, email support is limited >> to email notifications and there is limited if any support for email >> user participation. >> >> Third, there should be better support for moderator tools and >> management >> of the mailing list. See above for complaints about problems with the >> mailman2 system. >> >> Burdens of running your own mailing list and email server >> ========================================================= >> >> If you have never operated your own MTA you have no idea how difficult >> it is to keep secure and functional in the face of numerous challenges >> to deliverability. Anti-spam filtering is essential to prevent >> forwarding spam. The moment you forward even a single spam message you >> run the risk of the server IP address being added to blacklists. >> >> The problem of spam filtering is so bad that most IP addresses are >> presumed guilty even if they have no prior spam history, such as if >> their network or subnetwork had spam issues in the past. >> >> Even if you put unlimited time into managing your own email server, >> other people may not accept your email. Or you make one mistake, and >> then you get into permanent blacklists and it's hard to remove. The >> spam >> problem is so bad that most IPs are automatically on a >> guilty-until-proven-innocent blacklist. >> >> Often there is nothing you can do to get server IP addresses removed >> from spam blacklists or from "bad reputation" lists. >> >> Ironically, hashcash-style proof-of-work stamps to prevent spam are an >> appealing solution but not widely used in this community. Or anywhere. >> >> Infinite rejection or forwarding loops happen. They often need to be >> detected through vigilance and require manual sysadmin intervention to >> solve. >> >> Bitcoin's dev lists being hosted alongside other Open Source projects >> was previously protective. If that mailing list server became >> blacklisted there were a lot of other people who would notice and >> complain. If we run a Bitcoin-specific mail server we are on our own. >> 100% of the administrative burden falls upon our own people. There is >> also nothing we can do if some unknown admin decides they don't like >> us. >> >> Options >> ======= >> >> Web forums are an interesting option, but often don't have good email >> user integration. At most you can usually hope for email notifications >> and an ability to reply by email. It changes the model of the >> community >> from push (email) to pull (logging into a forum to read). RSS feeds >> can >> help a little bit. >> >> Many other projects have moved from mailing lists to forums (eg >> https://discuss.python.org/ <https://discuss.python.org/> - see >> https://lwn.net/Articles/901744/ <https://lwn.net/Articles/901744/> ; >> or >> https://ethresear.ch/ <https://ethresear.ch/>), which seem easier to >> maintain and moderate, and can have lots of advanced features beyond >> plaintext, maybe-threading and maybe-HTML-markup. >> >> Who would host the forum? Would there be agreement around which forum >> software to use or which forum host? What about bitcointalk.org >> <http://bitcointalk.org> or delvingbitcoin.org >> <http://delvingbitcoin.org>? There are many options available. Maybe >> what we actually want isn't so much a discussion forum, as an 'arxiv >> of >> our own' where anons can post BIP drafts and the like? >> >> Given the problems with mailman2, and the decline of email communities >> in general, it seems that moving to mailman3 would not be a viable >> long-term option. This leaves us with Google Groups or groups.io >> <http://groups.io> as two remaining options. >> >> groups.io <http://groups.io> is an interesting option: they are a paid >> service that implements email communities along with online web forum >> support. However, their public changelog indicates it has been a few >> years since their last public change. They might be a smaller company >> and it is unclear how long they will be around or if this would be the >> right fit for hosting sometimes contentious bitcoin development >> discussions... >> >> Google Groups is another interesting option, and comes with different >> tradeoffs. It's the lowest effort to maintain option, and has both an >> email interface and web forum interface. Users can choose which mode >> they want to interact with. >> >> For the Google Groups web interface, you can use it with a non-gmail >> account, but you must create a Google Account which is free to do. it >> does not require any personal information to do so. This also allows >> you >> to add 2FA. Non-gmail non-google users are able to subscribe and post >> email from their non-gmail non-google email accounts. Tor seems to >> work >> for the web interface. >> >> Will Google shut it down, will they cut us off, will they shut down >> non-google users? The same problem exists with other third-party >> hosts. >> >> The moderation capabilities for Google Groups and groups.io >> <http://groups.io> seem to be comparable. It seems more likely that >> Google Groups will be able to handle email delivery issues far better >> than a small resource-constrained operation like groups.io >> <http://groups.io>. ((During feedback for this draft, luke-jr >> indicates >> that Google Workspaces has been known to use blacklisted IPs for >> business email!)) >> >> On the other hand, groups.io <http://groups.io> is a paid service and >> you get what you pay for... hopefully? >> >> Finally, another option is to do literally nothing. It's less work >> overall. Users can switch to forums or other websites, or private >> one-on-one communication. It would remove a point of >> semi-centralization >> from the bitcoin ecosystem. It would hasten ossification, but on the >> other hand it would hasten ossification and this could be a negative >> too. Moderators would be less of a target. >> >> Unfortunately, by doing nothing, there would be no more widely used >> group email communication system between bitcoin developers. >> Developers >> become less coordinated, mayhem and chaos as people go to different >> communication platforms, a divided community is more vulnerable, etc. >> BIP1 and BIP2 would need to be revised for other venues. >> >> The main categories of what to move to are: web forums, mailing lists, >> and hybrids of those two options. Most everything is either >> self-hosted >> or you pay someone else to host it. It's kind of the same problem >> though. It largely depends on how good is the software and >> unfortunately >> running your own MTA that forwards mail is not a good option. >> >> Going forward >> =========== >> >> We'd like to invite feedback and proposals from the community, and see >> what options are available. One potential option is a migration to >> Google Groups, but we're open to ideas at this point. We apologize for >> any inconvenience this disruption has caused. >> >> Bitcoin-dev mailing list moderation team >> >> Bryan Bishop >> Ruben Somsen >> Warren Togami >> various others. >> >> -- >> - Bryan >> https://twitter.com/kanzure <https://twitter.com/kanzure> > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
On Tue, Nov 07, 2023 at 11:08:58PM +0000, Peter Todd via bitcoin-dev wrote:
Quoted text (11 lines)
> On Tue, Nov 07, 2023 at 09:37:22AM -0600, Bryan Bishop via bitcoin-dev wrote: > > Anti spam has been an issue for the moderators. It's relentless. Without > > access to the underlying server, it has been difficult to fight spam. There > > is some support for filters in mailman2 but it's not great. > > Since this is a technical mailing list it would be fine to require people to > pay a non-refundable anti-spam fee, eg via lightning, to gain the ability to > send messages. While this would require some custom software, it's probably > even possible to implement this if a third party is used for hosting, provided > they have some kind of API. >
Could be done but it's overkill. Running your own mail server is not that scary as the initial post makes it out to be. Sending out spam is problematic only when you allow imap/pop3 access which is not the case when running a mailling list.
My suggestion is to get the Bitcoin CEO hook us up on https://lists.freebsd.org Failling that I volunteer to get the same setup going for a community run project.
On Tue, Nov 07, 2023 at 06:14:23PM +0000, Andrew Chow via bitcoin-dev wrote:
Quoted text (5 lines)
>Hi Dan, > >I don't think nostr would be a suitable replacement for the mailing >list, although this opinion is biased by the fact that I do not use >nostr and find it to be uninteresting.
email-like functionality over nostr isn't really explored yet, but it is something I'm interesting in. So I agree nostr isn't a suitable email replacement *yet*.
From my limited understanding of how nostr works, it's not clear to me
Quoted text (2 lines)
>how a distributed system that uses message broadcast would work in the >same way as a mailing list.
My idea was to have a mailing list relay, the only thing missing is To: and Cc: tags on notes so that the relay can reject notes not destined for the mailing list
Quoted text (2 lines)
>How would people "subscribe"? How would archives be searched or >otherwise be available to people who are not on nostr?
You would subscribe by connecting to the relay and pulling down the notes. your client could cache notes and only pull new ones.
Quoted text (3 lines)
>How do you distinguish and filter between legitimate dev posts >intended for discussion and random crap and shitposting as shows up on >social media?
You would need to have metadata on the note that specifies that the note is destined for that specific mailing list relay (To, Cc, etc). Then the client sending the message can send it to that specific relay during note composition. Again, this is different than then current model that exists with social networking clients designed for blasting your note to as many people as possible.
Quoted text (6 lines)
>I also don't think that long form text on nostr (or any similar >platform) can sufficiently replace email. None of these things seem to >contain a way to have a separate subject line as email does. Subjects >are immensely important for me as it provides a quick and easy way to >filter out things I don't care about reading. I don't want to have read >something in before I can decide that I don't care about reading it.
Subject lines already exist in nostr and are a part of some email-like clients like https://github.com/unclebob/more-speech . it's just a tag like every other piece of metadata.
Quoted text (8 lines)
>In general, I strongly prefer email, or a platform that has email as a >first class user interface, over platforms such as nostr, matrix, or web >forums. Email is universal - everyone has one and everyone knows how it >works. It dramatically lowers the barrier of entry. Having to make an >account somewhere or download some specific client in order to >participate will simply result in only the most dedicated participating. >Development in open source must be an open process and the barriers to >entry should be low.
I definitely prefer email at the moment as well, but it is also a pain in the ass to run email infra. As someone who runs both email servers and nostr relays I can say nostr is much more pleasant.
So yeah, it's a bit too early for a nostr replacement, but it's definitely possible, and you get proper cryptographic identities and schnorr signed notes which is a bonus. For dealing with spam you could have a sat entrance fee via lightning. I will start looking into this!
Cheers,
Will
What about using Signet, or some separate P2P network, to handle all of that?
1. All e-mails could be sent in a pure P2P way, just each "mailing list node" would receive it, and include to its mempool. 2. The inclusion of some message would be decided by signing a block. Moderators would pick the proper messages, and publish them by broadcasting a new block to all nodes. 3. Each message will be signed by some public key. It could be changed each time, or even derived from some HD wallet. Only those owning "master public keys" would know, which messages were sent by the same person. 4. The time of the block could be much longer than 10 minutes. It could be for example one hour, one day, or even longer. Or, the commitment to all of that could be just included "every sometimes" to the existing Signet chain, because it would take no additional on-chain bytes, and can be easily done in the coinbase transaction. 5. If there will be too much spam in the mempool, then hashcash-based Proof of Work can be used to filter messages. Instead of fee-based filtering, it could be Proof-of-Work-based filtering. Even better: because of "master public keys", the regular participants could be allowed anyway, without providing additional Proof of Work. Their signature would be sufficient in that case. 6. The code is almost there. Maybe there are even altcoins, designed specifically for storing data, and we could just use them? 7. This kind of decision would push things like Silent Payments forward. Because then, you could develop scanners, to know, who wrote which message. You could enter some "master public key", scan the whole chain, and find out all messages written by that particular participant. 8. It would push commitments forward. Because then, it would be possible to send some message to the "P2P mailing list network", and reveal it later. Of course, it is not mandatory to accept commitments at all, which means, they could be easily disabled, if they would be misused. Or we could start with no commitments, and introduce them later if needed. 9. Because Signet challenge can contain some multisig, or even some Taproot address, there will be no issue with using the same password to access the moderation panel. Also, in that case, it is possible to prove later, which moderator accepted which message. And also, it is still possible to use some shared key, if revealing that is not desirable, or even it is possible to easily reach "approved by all moderators" messages, because their Schnorr signatures could be combined. Also, any K-of-N multisig can be battle-tested in that way.
So, I can see two options: reusing some existing P2P network, or making a new one, designed specifically for handling mailing list messages in a pure P2P way. I guess we can try some existing chains first, and if there is no promising altcoin, or if we don't want to be associated with any altcoin, then our own Signet-like network could solve it.
Thanks for the write up and thanks to the bitcoin-dev mailing list moderation team for their work along the years.
If we can pick up a communication platform where platform moderators / infra maintainers have low-risk of being targeted by subpoena + gag order or "injonction administrative" (the equivalent in some civil law systems) due to lack of moderators discretionary decisions, I think this is a good outcome.
I don't know of such a communication platform or set of protocols as of today. Nostr is promising though realistically weak until half a decade of work is poured in.
Personally, I'll be more present on the Delving Bitcoin forum, though it sounds more a temporary solution than a long-term ideal. Being hosted by kernels or other old open-sources project mailing list infra sounds like a good idea.
Best, Antoine
Le mar. 7 nov. 2023 à 15:37, Bryan Bishop via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> a écrit :
Quoted text (277 lines)
> Hello, > > We would like to request community feedback and proposals on the future of > the mailing list. > > Our current mailing list host, Linux Foundation, has indicated for years > that they have wanted to stop hosting mailing lists, which would mean the > bitcoin-dev mailing list would need to move somewhere else. We temporarily > avoided that, but recently LF has informed a moderator that they will cease > hosting any mailing lists later this year. > > In this email, we will go over some of the history, options, and invite > discussion ahead of the cutoff. We have some ideas but want to solicit > feedback and proposals. > > Background > ========== > > The bitcoin-dev mailing list was originally hosted on Sourceforge.net. The > bitcoin development mailing list has been a source of proposals, analysis, > and developer discussion for many years in the bitcoin community, with many > thousands of participants. Later, the mailing list was migrated to the > Linux Foundation, and after that OSUOSL began to help. > > Linux Foundation first asked us to move the mailing list in 2017. They > internally attempted to migrate all LF mailing lists from mailman2 to > mailman3, but ultimately gave up. There were reports of scalability issues > with mailman3 for large email communities. Ours definitely qualifies as.. > large. > > 2019 migration plan: LF was to turn off mailman and all lists would > migrate to the paid service provider groups.io. Back then we were given > accounts to try the groups.io interface and administration features. > Apparently we were not the only dev community who resisted change. To our > surprise LF gave us several years of reprieve by instead handing the > subdomain and server-side data to the non-profit OSUOSL lab who instead > operated mailman2 for the past ~4 years. > > OSUOSL has for decades been well known for providing server infrastructure > for Linux and Open Source development so they were a good fit. This however > became an added maintenance burden for the small non-profit with limited > resources. Several members of the Bitcoin dev community contributed funding > to the lab in support of their Open Source development infrastructure > goals. But throwing money at the problem isn’t going to fix the ongoing > maintenance burden created by antiquated limitations of mailman2. > > Permalinks > ========== > > Linux Foundation has either offered or agreed to maintain archive > permalinks so that content of historic importance is not lost. Fortunately > for us while lists.linuxfoundation.org mailman will go down, they have > agreed the read-only pipermail archives will remain online. So all old URLs > will continue to remain valid. However, the moderators strongly advise that > the community supplements with public-inbox instances to have canonical > archive urls that are separate from any particular email software host. > > Public-Inbox > ============ > > https://public-inbox.org/README.html > > “Public Inbox” decentralized archiving - no matter what mailing list > server solution is used, anyone can use git to maintain their own mailing > list archive and make it available to read on the web. > > Public Inbox is a tool that you can run yourself. You can transform your > mbox file and it makes it browsable and viewable online. It commits every > post to a git repository. It's kind of like a decentralized mail archiving > tool. Anyone can publish the mail archive to any web server they wish. > > We should try to have one or more canonical archives that are served using > public-inbox. But it doesn't matter if these are lost because anyone else > can archive the mailing list in the same way and re-publish the archives. > > These git commits can also be stamped using opentimestamps, inserting > their hashes into the bitcoin blockchain. > > LKML mailing list readers often use public-inbox's web interface, and they > use the reply-to headers to populate their mail client and reply to threads > of interest. This allows their reply to be properly threaded even if they > were not a previous subscriber to that mailing list to receive the headers. > > public-inbox makes it so that it doesn't really matter where the list is > hosted, as pertaining to reading the mailing list. There is still a > disruption if the mailing list goes away, but the archives live on and then > people can post elsewhere. The archive gets disconnected from the mailing > list host in terms of posting. We could have a few canonical URLs for the > hosts, separate from the mailing list server. > > mailman problems > ================ > > Over the years we have identified a number of problems with mailman2 > especially as it pertains to content moderation. There are presently a > handful of different moderators, but mailman2 only has a single password > for logging into the email management interface. There are no moderator > audit logs to see which user (there is no concept of different users) acted > on an email. There is no way to mark an email as being investigated by one > or more of the moderators. Sometimes, while investigating the veracity of > an email, another moderator would come in and approve a suspect email by > accident. > > Anti spam has been an issue for the moderators. It's relentless. Without > access to the underlying server, it has been difficult to fight spam. There > is some support for filters in mailman2 but it's not great. > > 100% active moderation and approval of every email is unsustainable for > volunteer moderators. A system that requires moderation is a heavy burden > on the moderators and it slows down overall communication and productivity. > There's lots of problems with this. Also, moderators can be blamed when > they are merely slow while they are not actually censoring. > > Rejection emails can optionally be sent to > bitcoin-dev-moderation@lists•ozlabs•org but this is an option that a > moderator has to remember to type in each time. > > Not to mention numerous bugs and vulnerabilities that have accumulated > over the years for relatively unmaintained software. (Not disclosed here) > > Requirements and considerations > =============================== > > Looking towards the future, there are a number of properties that seem to > be important for the bitcoin-dev mailing list community. First, it is > important that backups of the entire archive should be easy for the public > to copy or verify so that the system can be brought up elsewhere if > necessary. > > Second, there seems to be demand for both an email threading interface > (using mailing list software) as well as web-accessible interfaces (such as > forum software). There seems to be very few options that cater to both > email and web. Often, in forum software, email support is limited to email > notifications and there is limited if any support for email user > participation. > > Third, there should be better support for moderator tools and management > of the mailing list. See above for complaints about problems with the > mailman2 system. > > Burdens of running your own mailing list and email server > ========================================================= > > If you have never operated your own MTA you have no idea how difficult it > is to keep secure and functional in the face of numerous challenges to > deliverability. Anti-spam filtering is essential to prevent forwarding > spam. The moment you forward even a single spam message you run the risk of > the server IP address being added to blacklists. > > The problem of spam filtering is so bad that most IP addresses are > presumed guilty even if they have no prior spam history, such as if their > network or subnetwork had spam issues in the past. > > Even if you put unlimited time into managing your own email server, other > people may not accept your email. Or you make one mistake, and then you get > into permanent blacklists and it's hard to remove. The spam problem is so > bad that most IPs are automatically on a guilty-until-proven-innocent > blacklist. > > Often there is nothing you can do to get server IP addresses removed from > spam blacklists or from "bad reputation" lists. > > Ironically, hashcash-style proof-of-work stamps to prevent spam are an > appealing solution but not widely used in this community. Or anywhere. > > Infinite rejection or forwarding loops happen. They often need to be > detected through vigilance and require manual sysadmin intervention to > solve. > > Bitcoin's dev lists being hosted alongside other Open Source projects was > previously protective. If that mailing list server became blacklisted there > were a lot of other people who would notice and complain. If we run a > Bitcoin-specific mail server we are on our own. 100% of the administrative > burden falls upon our own people. There is also nothing we can do if some > unknown admin decides they don't like us. > > Options > ======= > > Web forums are an interesting option, but often don't have good email user > integration. At most you can usually hope for email notifications and an > ability to reply by email. It changes the model of the community from push > (email) to pull (logging into a forum to read). RSS feeds can help a little > bit. > > Many other projects have moved from mailing lists to forums (eg > https://discuss.python.org/ – see https://lwn.net/Articles/901744/ ; or > https://ethresear.ch/), which seem easier to maintain and moderate, and > can have lots of advanced features beyond plaintext, maybe-threading and > maybe-HTML-markup. > > Who would host the forum? Would there be agreement around which forum > software to use or which forum host? What about bitcointalk.org or > delvingbitcoin.org? There are many options available. Maybe what we > actually want isn’t so much a discussion forum, as an 'arxiv of our own' > where anons can post BIP drafts and the like? > > Given the problems with mailman2, and the decline of email communities in > general, it seems that moving to mailman3 would not be a viable long-term > option. This leaves us with Google Groups or groups.io as two remaining > options. > > groups.io is an interesting option: they are a paid service that > implements email communities along with online web forum support. However, > their public changelog indicates it has been a few years since their last > public change. They might be a smaller company and it is unclear how long > they will be around or if this would be the right fit for hosting sometimes > contentious bitcoin development discussions... > > Google Groups is another interesting option, and comes with different > tradeoffs. It's the lowest effort to maintain option, and has both an email > interface and web forum interface. Users can choose which mode they want to > interact with. > > For the Google Groups web interface, you can use it with a non-gmail > account, but you must create a Google Account which is free to do. it does > not require any personal information to do so. This also allows you to add > 2FA. Non-gmail non-google users are able to subscribe and post email from > their non-gmail non-google email accounts. Tor seems to work for the web > interface. > > Will Google shut it down, will they cut us off, will they shut down > non-google users? The same problem exists with other third-party hosts. > > The moderation capabilities for Google Groups and groups.io seem to be > comparable. It seems more likely that Google Groups will be able to handle > email delivery issues far better than a small resource-constrained > operation like groups.io. ((During feedback for this draft, luke-jr > indicates that Google Workspaces has been known to use blacklisted IPs for > business email!)) > > On the other hand, groups.io is a paid service and you get what you pay > for... hopefully? > > Finally, another option is to do literally nothing. It's less work > overall. Users can switch to forums or other websites, or private > one-on-one communication. It would remove a point of semi-centralization > from the bitcoin ecosystem. It would hasten ossification, but on the other > hand it would hasten ossification and this could be a negative too. > Moderators would be less of a target. > > Unfortunately, by doing nothing, there would be no more widely used group > email communication system between bitcoin developers. Developers become > less coordinated, mayhem and chaos as people go to different communication > platforms, a divided community is more vulnerable, etc. BIP1 and BIP2 would > need to be revised for other venues. > > The main categories of what to move to are: web forums, mailing lists, and > hybrids of those two options. Most everything is either self-hosted or you > pay someone else to host it. It's kind of the same problem though. It > largely depends on how good is the software and unfortunately running your > own MTA that forwards mail is not a good option. > > Going forward > =========== > > We'd like to invite feedback and proposals from the community, and see > what options are available. One potential option is a migration to Google > Groups, but we're open to ideas at this point. We apologize for any > inconvenience this disruption has caused. > > > Bitcoin-dev mailing list moderation team > > Bryan Bishop > Ruben Somsen > Warren Togami > various others. > > -- > - Bryan > https://twitter.com/kanzure > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On Tue, 2023-11-07 at 09:37 -0600, Bryan Bishop via bitcoin-dev wrote: Google Groups is another interesting option,
I don't think I'm the only person on this list that is strongly opposed to using google for anything. They are too big and they have their hand in everything, and their eyes (and analytics) on everything.
I remember when there were virtually no gmail email addresses that posted to this list. Suddenly in 2020 or 2021, we had an influx of gmail subscribers and posters. That didn't escape me then and it is not lost on me now.
Email is great for public discussion for many reasons. The fact that everyone gets a copy of the data, there is no single central authority that can edit emails once they have been sent out. Anyone can archive email messages, they can generally store or publish the data anywhere they like. That is not the case with web forum content.
I like the lightning anti-spam fee idea. That would encourage me to finally adopt lightning, and it would, I'm sure, produce some interesting results for the list.
I don't think email should be out of the question. Does anyone besides kanzure@gmail•com think that sticking with email is out of the question?
Let's do what's necessary to stick with email.
Hi Overthefalls,
+1
Using google for bitcoin mailing list is not good. It feels embarrassing that some developers that built and maintained the only decentralized network used to settle uncensored payments and some of them even working on nostr, can't build their own mailing list which is better than present mailing list. I have some ideas but it seems the influential developers have already decided and wont accept anything.
Nostr can be used to build a mailing list which also allows anyone to send emails apart from publishing events from different clients. We just need a new NIP so that nostr relays understand its a different event. There can be multiple front end with different levels of moderation to hide some emails and ultimately one will be used the most. It can use multiple relays and relays share some information in NIP 11 which can include an email address.
/dev/fd0 floppy disk guy
Sent with [Proton Mail](https://proton.me/) secure email.
On Monday, November 13th, 2023 at 8:35 PM, Overthefalls via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (15 lines)
> On Tue, 2023-11-07 at 09:37 -0600, Bryan Bishop via bitcoin-dev wrote: > >> Google Groups is another interesting option, > > I don't think I'm the only person on this list that is strongly opposed to using google for anything. They are too big and they have their hand in everything, and their eyes (and analytics) on everything. > > I remember when there were virtually no gmail email addresses that posted to this list. Suddenly in 2020 or 2021, we had an influx of gmail subscribers and posters. That didn't escape me then and it is not lost on me now. > > Email is great for public discussion for many reasons. The fact that everyone gets a copy of the data, there is no single central authority that can edit emails once they have been sent out. Anyone can archive email messages, they can generally store or publish the data anywhere they like. That is not the case with web forum content. > > I like the lightning anti-spam fee idea. That would encourage me to finally adopt lightning, and it would, I'm sure, produce some interesting results for the list. > > I don't think email should be out of the question. Does anyone besides kanzure@gmail•com think that sticking with email is out of the question? > > Let's do what's necessary to stick with email.
I find Google Groups especially repugnant not not only because what has already been mentioned, but Google Groups has a quite clunky and annoying user interface that makes it difficult for me to find anything or interest in there.
Usenet was migrated to Google Groups for some reason, and it's very difficult to search for anything of particular interest using that site.
Not to mention that Google Groups also contains a larger amount of spam (w.r.t value), so arguably the moderation burden will be higher.
It is necessary to try to find a way to keep the discussion on a mail server, since a migration off of it will render many users' email clients useless for this purpose.
- Ali
Quoted text (32 lines)
> On Mon, 13 Nov 2023 18:51:26 +0000, alicexbt <alicexbt@protonmail•com> wrote: > > Hi Overthefalls, > > +1 > > Using google for bitcoin mailing list is not good. It feels embarrassing that some developers that built and maintained the only decentralized network used to settle uncensored payments and some of them even working on nostr, can't build their own mailing list which is better than present mailing list. I have some ideas but it seems the influential developers have already decided and wont accept anything. > > Nostr can be used to build a mailing list which also allows anyone to send emails apart from publishing events from different clients. We just need a new NIP so that nostr relays understand its a different event. There can be multiple front end with different levels of moderation to hide some emails and ultimately one will be used the most. It can use multiple relays and relays share some information in NIP 11 which can include an email address. > > /dev/fd0 > floppy disk guy > > Sent with Proton Mail secure email. > > On Monday, November 13th, 2023 at 8:35 PM, Overthefalls via bitcoin-dev bitcoin-dev@lists•linuxfoundation•org wrote: > > > On Tue, 2023-11-07 at 09:37 -0600, Bryan Bishop via bitcoin-dev wrote: > > > > > Google Groups is another interesting option, > > > > I don't think I'm the only person on this list that is strongly opposed to using google for anything. They are too big and they have their hand in everything, and their eyes (and analytics) on everything. > > > > I remember when there were virtually no gmail email addresses that posted to this list. Suddenly in 2020 or 2021, we had an influx of gmail subscribers and posters. That didn't escape me then and it is not lost on me now. > > > > Email is great for public discussion for many reasons. The fact that everyone gets a copy of the data, there is no single central authority that can edit emails once they have been sent out. Anyone can archive email messages, they can generally store or publish the data anywhere they like. That is not the case with web forum content. > > > > I like the lightning anti-spam fee idea. That would encourage me to finally adopt lightning, and it would, I'm sure, produce some interesting results for the list. > > > > I don't think email should be out of the question. Does anyone besides kanzure@gmail•com think that sticking with email is out of the question? > > > > Let's do what's necessary to stick with email.
Hi floppy disk guy, thanks for prompting me to look closer at Nostr, it's very interesting.
I hope that whatever solution is chosen doesn't involve handing power over to a centralized entity that wants collect as much information on every living person as possible, and lock everyone and everything into using it's services forever.
On Mon, 2023-11-13 at 18:51 +0000, alicexbt wrote:
Quoted text (77 lines)
> Hi Overthefalls, > > > +1 > > > Using google for bitcoin mailing list is not good. It feels > embarrassing that some developers that built and maintained the only > decentralized network used to settle uncensored payments and some of > them even working on nostr, can't build their own mailing list which > is better than present mailing list. I have some ideas but it seems > the influential developers have already decided and wont accept > anything. > > Nostr can be used to build a mailing list which also allows anyone to > send emails apart from publishing events from different clients. We > just need a new NIP so that nostr relays understand its a different > event. There can be multiple front end with different levels of > moderation to hide some emails and ultimately one will be used the > most. It can use multiple relays and relays share some information in > NIP 11 which can include an email address. > > > /dev/fd0 > floppy disk guy > > > > > > > > > Sent with Proton Mail secure email. > > > > > > On Monday, November 13th, 2023 at 8:35 PM, Overthefalls via > bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote: > > > > > On Tue, 2023-11-07 at 09:37 -0600, Bryan Bishop via > > bitcoin-dev wrote: > > > Google Groups is another interesting option, > > > > I don't think I'm the only person on this list that is strongly > > opposed to using google for anything. They are too big and they > > have their hand in everything, and their eyes (and analytics) on > > everything. > > I remember when there were virtually no gmail email addresses that > > posted to this list. Suddenly in 2020 or 2021, we had an influx of > > gmail subscribers and posters. That didn't escape me then and it is > > not lost on me now. > > Email is great for public discussion for many reasons. The fact > > that everyone gets a copy of the data, there is no single central > > authority that can edit emails once they have been sent out. Anyone > > can archive email messages, they can generally store or publish the > > data anywhere they like. That is not the case with web forum > > content. > > I like the lightning anti-spam fee idea. That would encourage me to > > finally adopt lightning, and it would, I'm sure, produce some > > interesting results for the list. > > I don't think email should be out of the question. Does anyone > > besides kanzure@gmail•com think that sticking with email is out of > > the question? > > Let's do what's necessary to stick with email. > > > > > > > > > > > > > >
Hi all,
It looks like there are only a few mailing lists left on https://lists.linuxfoundation.org/mailman/listinfo and all of the remaining ones are using Mailman version 2.1.15, which is not the current version - https://www.gnu.org/software/mailman/
Was there any decision made on where to move the bitcoin-dev mailing list to?
Thanks,
Brad
On 2023-11-11 02:54, vjudeu via bitcoin-dev wrote:
Quoted text (17 lines)
> What about using Signet, or some separate P2P network, to handle all of that? > > 1. All e-mails could be sent in a pure P2P way, just each "mailing list node" would receive it, and include to its mempool. > 2. The inclusion of some message would be decided by signing a block. Moderators would pick the proper messages, and publish them by broadcasting a new block to all nodes. > 3. Each message will be signed by some public key. It could be changed each time, or even derived from some HD wallet. Only those owning "master public keys" would know, which messages were sent by the same person. > 4. The time of the block could be much longer than 10 minutes. It could be for example one hour, one day, or even longer. Or, the commitment to all of that could be just included "every sometimes" to the existing Signet chain, because it would take no additional on-chain bytes, and can be easily done in the coinbase transaction. > 5. If there will be too much spam in the mempool, then hashcash-based Proof of Work can be used to filter messages. Instead of fee-based filtering, it could be Proof-of-Work-based filtering. Even better: because of "master public keys", the regular participants could be allowed anyway, without providing additional Proof of Work. Their signature would be sufficient in that case. > 6. The code is almost there. Maybe there are even altcoins, designed specifically for storing data, and we could just use them? > 7. This kind of decision would push things like Silent Payments forward. Because then, you could develop scanners, to know, who wrote which message. You could enter some "master public key", scan the whole chain, and find out all messages written by that particular participant. > 8. It would push commitments forward. Because then, it would be possible to send some message to the "P2P mailing list network", and reveal it later. Of course, it is not mandatory to accept commitments at all, which means, they could be easily disabled, if they would be misused. Or we could start with no commitments, and introduce them later if needed. > 9. Because Signet challenge can contain some multisig, or even some Taproot address, there will be no issue with using the same password to access the moderation panel. Also, in that case, it is possible to prove later, which moderator accepted which message. And also, it is still possible to use some shared key, if revealing that is not desirable, or even it is possible to easily reach "approved by all moderators" messages, because their Schnorr signatures could be combined. Also, any K-of-N multisig can be battle-tested in that way. > > So, I can see two options: reusing some existing P2P network, or making a new one, designed specifically for handling mailing list messages in a pure P2P way. I guess we can try some existing chains first, and if there is no promising altcoin, or if we don't want to be associated with any altcoin, then our own Signet-like network could solve it. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev