March 21, 2024: Meta announces that Threads ActivityPub integration is now available in the United States, Canada and Japan.
Jan 8th 2025:
dnlbrnds: Good day everybody — I think this has been discussed when Threads went live, but with the recent developments around Meta, do you think we shall reconsider de-federating it?
3wc: I have to say the previous decisions seemed like a textbook case of how decisionmaking processes are vulnerable to how the question is framed, and it would be great to try to correct that this time around
it’s clear to me that Threads should be defederated based on our existing policy, so I think the proposal should be framed as "Should we make an exception to our policy for Meta", which I think is the correct & fair allocation of "status quo" / "do nothing" voting
flancian suggests a vote would be needed for taking action and "I’d recommend interested people ideally get together to agree on the phasing and then start one"
dnlbrnds and 3wc discuss co-drafting a proposal
Jan 9th, 2025:
Flancian: Thanks for the context and sharing your view! FWIW Meta does have policies still and claims to enforce them, the question then seems to be whether the policies are good, mediocre or actively harmful (the "encouraging abuse" argument). Abuse to members of our instance has not materialized from threads.net yet AFAICT; I can review moderation reports to be sure if people think that would be an interesting data point.
3wc: [are you saying] that your interpretation is that if a server’s policy only bans some hate speech (while explicitly allowing other types) then it does not "[lack] policies .. to deal with hate speech"?
Flancian: [..] I was trying to say that Meta does seem to be defining hate speech and is apparently "just" choosing to exclude some classes of speech from this category, which is different than not caring about hate speech at all like e.g. nazi instances do.
3wc: I have read many Nazi instances’ policies and a significant number are similar to Facebook’s - some hate speech is disallowed, some is explicitly allowed
8HJYUNMMMMMrts a thread with the CWG
16:39 UTC foolishowl: It sounds like several of our moderators think it’s best to have a vote on this, and I haven’t seen any arguments against doing so. Unfortunately I’m bogged down at work, so I can’t do a proper job drafting it at the moment, but I’ll work on it when I can.
20:58 UTC Flancian: I will check back later tonight and run the vote by default, then we can go further in Loomio? I think it’s a good opportunity for people to express their views on what is happening. I hope the community finds the time to participate widely.
21:25 UTC Flancian: PSA: https://www.loomio.com/d/u1OkUA6M/clarity-on-stance-with-regards-to-threads/30 is the vote. please vote at your earliest convenience, but also optionally take the time to consider what you would like to publish on such an occasion; I think history will likely look back on both corporations and the commons at this moment. We have six days as per our governance template.
22:00 UTC: flancian and 3wc have a 1:1 discussion about the vote
3wc: "as i said in DM, I will take you on good faith that that was your conscious intention. I will also observe that your choice of framing has made it more likely that the vote would reach your desired outcome and placed the burden on more-marginalised people to advocate for ourselves to maintain our existing defences - and encourage you to consider what subconscious bias might also have been a factor in this choice"
12:44 UTC Flancian: 20% is also significant enough that, if it holds, we should probably discuss as a community how exactly we go through enacting the suspend to make sure people don’t lose significant data (as per the message above; but that is only one particular implementation plan).
January 14, 2025:
Flancian: I’ve now called this out to the CWG and TWG so we can plan accordingly. the CWG is meeting on the 23rd and this is a topic of discussion (how to defederate if the vote holds). so people should not expect a suspend action before then, maybe before EOM. if anybody has any issues with this, please raise.
3wc: 85% of people who voted (in the second poll) have actively disagreed with making an exception to the policy based on scale. But you’re pushing for an exception to the policy based on scale?
if people want "how many ‘follow’ relationships exist between social.coop and the instance?" to be a factor in the implementation of the Federation abuse policy then let’s get a proposal for it to be added?
Flancian: "Pushing" is forceful language that does not represent what I feel.
Flancian: Finally, can we acknowledge I voted to suspend and not to grant an exception to our policies and I’m acting under the working assumption the first vote will pass and the second will not? In which way do you think I am misreading, mishandling or worsening the situation
3wc ~ they/them? Thank you for clarifying.
Timeline highlights / overview
F: Feeling very invested in things going into the situation. Suggested vote on the 8th, saw people worrying. 9th read more reaction, started changing my mind from "limit" based on lack of practical impact, to "suspend" based on emotional impact. 10th felt like the issue escalated. My POV: asked to wait. Felt like people were not OK with that. Suggested foolishowl create a thread. Didn’t see pushback against calling a vote. Started a vote 1.5 hours after sending draft. Felt like reaction that took place after was not as constructive as it could have been. Felt responsibility as CWG member to start the vote. Did not feel in a good position to manage the conflict. Assumptions about personal motivations. Led to feeling disappointed.
C: Felt disappointed as well when I saw the vote framed in this way. Felt that future at coop was uncertain. Suspicious about the shortness of the draft review. Felt unseen/heard.
Clarifying questions
About past proposals in this space and their issues
Next step here is still to choose a VPS size/configuration in iocoop.org
Maybe decouple from instance restore question in the following way
We could ‘cook’ a db restore that incorporates all tables but skips >90% of history, bringing down db size to something more manageable in a small instance?
Hosting costs
Let the FWG cost what we’re taking on (to budget for 2025)
And also try to move to a model in which we 1. have a good relationship with our hosting providers, which invoice us ideally on a known cadence and 2. have reduced risk of service disruption.
[[Dan]] too the day off work for daughter’s birthday
Top priorities
Thanks Dan for solving the downtime the other day!
What should we do so this is not necessary again?
Dan: we should wait and see (tm) until the 15th, see if it auto-pays off the balance
F: how to solve systemic issue, "someone" should pay it? suggestion: Finance Working Group, Caitlin seems OK with that. Raised it but not spoken enough to fully decide. Should be able to resolve before next payment.
D: Can change card. First one with Paypal, second with card, put card on file. Required a second factor for setup, but authentication error paying it second time. So - autopay might not work. Is there a non-individual card we can use? Otherwise fine to continue.
F: Makes sense to have a card for the co-op but not yet. Finance working group on it.
D: SEPA and Bank Transfer are other options besides CC and PayPal
A bank account could be easier for the coop as we have funds somewhere in OC?
Calix: experience from other projects is that, as long as we’re with Open Collective, it’s entirely up to our fiscal host. OC used to offer virtual cards but in a previous experience it just didn’t work — it wasn’t for this kind of payment of services. So it wouldn’t really help with the bureaucracy, and they don’t offer it anymore. We probably don’t have access to a bank account as a coop currently to be able to send a transfer directly. Any of the payment methods would be the same, individual pays and then gets reimbursed.
F: will report this to the organizing circle and report back in our next meeting.
they are looking for contributors for implementing governance extensions
design
coding
edumerco: been helping them in the interaction design front. a group of students contributed last year. currently they are focused on fixing performance issues, are using bounties.
next steps:
announce to the community -> the OC could do this?
Edumerco: studied biology, born and lives in Buenos Aires. Part of the Organizing Circle; part of an intentional community near Buenos Aires. A fan of cooperation :)
Caitlin Waddick: live in Vermont, part of the Finance Working Group. Part of a few cooperatives. Currently in more than one meeting context :)
Eduardo (Flancian): part of the TWF for 2 years, social.coop for 4. Joined the CWG, and take some shifts, and also involved in the organizing circle. Super into cooperatives, and work in big-tech as part of daily life (e.g. this is meeting no. 8 today). Also from Argentina.
Edsu (Ed Summers): worked as a software developer for… long :) Also studied libraries and archives at school, have a background in digital preservation. Working in Stanford U Library. Belonged to many coops throughout life, but this is the only one I’m currently active in. Live in Maryland.
Calix: pronouns they/them. Main gig is autonomic coop. Also part of a second worker coop, about to join a third. Part of a few coop federations. Super happy to be here. Been in the TWG for a ~year. Background is in development, science, infrastructure. Autonomic initiated [[coop cloud]], which we’re using for some of our services. Usually based in NYC but currently in Canada.
Q (edumerco): do you know people from Sutty? We have them in common?
A: yes!
Edumerco is also at Sutty :) Would love to welcome in Buenos Aires.
Kathe Todd-Brown: in University of Florida. Been on Mastodon for 2-3y. Previously at Twitter for ~15y in academic Twitter. Geek out with collaboration and organizational structures. Run several groups in different modalities.
Questions from/to the OC
Q (edumerco): how can we help one another? What do we need to know, how can we best support our work and serve our community?
Edsu: something that came up in the matrix chat ~2w ago. On how we organize communication and our meetings. E.g. this meeting we’re having now is something we have been missing, as we haven’t had as much visibility on what the OC is doing/working on.
Edumerco: currently the OC is a group that "catches" whatever doesn’t fall to other working groups. It currently works as per [[sociocracy]]. We are not a board/it is not our place to mandate how working groups operate. We set out to organize but not make decisions for others. We can help with: determining which decisions must be made; who should take them.
Edumerco: this round of meetings is about understanding best what each group does and how we can support you.
Edsu: in terms of gaps, one that comes to mind is about payment of bills — the hosting bill was left unpaid recently. Is paying the bill a TWG issue or a FWG issue? How to know?
Flancian: we can separate the bills in terms of what needs to be paid, from who is paying them. There are notes about this in OC meeting minutes in Loomio.
Caitlin: posts can be posted to OpenCollective and members of the Finance Working Group can approve them.
Flancian: now that we are closer to establishing the co-op as a legal entity is it possible to have a co-op credit card which could be used?
Caitlin: will look into this.
Thank you Caitlin!!!
Edumerco: on the opportunity of defining our scopes, sociocratically. E.g. the TWG can say "we take decisions on technology, operate the services, but (propose that) we don’t pay bills".
Together with defining scopes, we can define how we (working groups) interact.
Calix: on inter-circle communication, maybe we should raise the fact that we are at low capacity. The OC could maybe boost this and help with recruiting?
Edumerco: perfect example of how we can help with, yes. If we can describe the kind of knowledge and interests that a good candidate would have, we could go with that to the community.
Flancian: we’ll definitely discuss scope and role descriptions in the next TWG meeting.
Edsu: +1 to Calix. We sometimes run a bit thin; although people come and go. E.g. Akshay did a lot of significant work and then had to take a step back. But still swooped in and purged docker logs in the middle of the night recently :) So it’s difficult to estimate what our actual capacity is.
Edsu: there is a relatively large group of people who have access to servers but are not active. We need to determine when people lose access.
Edumerco: this working group can decide that. Also, have you considered an oncall rotation/some sort of subgroup that could do small response tasks?
Flancian: +1 on the onduty/oncall group, similar to the CWG model. There is the issue of oncall-aversion for some as several of us are already oncall for our day jobs, and the fact that the response SLO for technical issues needs to be somewhat low.
This can go well with the task to review access lists.
Edumerco: question about documentation. Is it up to date?
Edsu: we dropped some things when we moved to wiki.social.coop. There could be more work on this area.
Ed: Nathan Schneider messaged us/some of us to onboard a script that is already on git.coop to cross-post loomio activity to mastodon
How much would we want to surface in socialcoop@social.coop? Activity for all WGs? Or only some?
Flancian: there is a question of how to do it, and where (e.g. hypha). Could start with it running it locally. And then there is a question of what to post. This could be a good question to bring up to the organizing circle.
Bring up with the Organizing Circle in our meeting: what do we want to cross-post?
Decide on which approach to take here? From the N tools considered
onboarding new members
We could set up a session to discuss infrastructure?
Would love that!
Let’s agree over matrix? Default to sometime next week.
are registrations working ok?
They are ok-ish.
Sometimes work, mostly.
As we were meeting there was a question posted to Matrix "did you receive XXX signup? they are asking if we got it" and I think fixing this will create more assurance all around that we didn’t lose any applications.
But there are rough edges:
Sometimes the spam trap over-triggers
The feedback is lacking even when it processes a registration; it just dumps the user in join.social.coop’s root, and you go ‘huh’ / are unsure if it worked
Fixing this/upgrading would mean some project work in node.js
didn’t find anything online about "push" queue. most info about smaller instances. would be nice to find "recipes". how can we investigate mastodon to try to find where sidekiq jobs might be coming from?
cd social.coop/ansible
vi roles/social.coop/files/docker-compose.yml
. .env
ansible-playbook -CD rhizome.playbook.yml # check the diff in dry run
ansible-playbook -D rhizome.playbook.yml # -D means diff, omitting -C makes it stick
next steps
- decide on way forward:
- still go with the hypha + coop cloud plan and
- use hometown or
- upgrade/fix mastodon
- set up a new alpha server and deploy a variation of the ansible role there
- (need a new server to still have the reverse proxy setup work without breaking hypha)
- modify ansible recipe to deploy alpha to hypha without reverse proxy
- can we set up [[traefik]] to work as reverse proxy only and forward to a non-coop-cloud-managed container?
- [[calix]]: if you add traefik labels to the docker compose file, traefik would configure itself automatically https://git.coopcloud.tech/coop-cloud/mastodon/src/branch/main/compose.yml#L19-L20
- you can also provide a traefik configuration file to the coop cloud traefik https://docs.coopcloud.tech/operators/handbook/#proxying-apps-outside-of-co-op-cloud-with-traefik
Result of the 1k chars poll
Q: if we’re ‘hot patching’ the character limits, are we OK with the extra build time for the frontend?
[[calix]]: it’s a yarn build that might add a minute or so to every container start
we’re ‘up two weeks’ currently
but any time we restart the container during [[twg]] operations, we’ll be more disruptive
[[evan]] replace images with their actual alt text instead, more representative
host/offer bounty on accessibility features for mastodon as a coop
[[dphiffer]] inject UI code instead of messing the backend
file a proposal, any proposal, to the mastodon project — e.g. ‘disable images’ as a setting, or disable posting images without alt text? or prompt users to do it in every case.
TLDR: may 16 is too soon to ship any of this, or even test the proposed breakage safely currently — but we could "commit" to doing something for the October date
It seems more likely we can get community + tech stack agreemend with such an effort by October (the second day proposed)
Pros and cons of breaking this feature for this campaign; S3 complications likely.
[[calix]] it might actually turn out that self-hosted loomio is harder to use, as some people may log in to loomio for more than one community
F: current default is to not run other services
[[jamie]] it might be useful to pin these to [[user journeys]] for members of the community. Would love to join that.
[[flancian]] we can try to keep a map of project:person who is interested and notify when there’s activity.
[[flancian]][[twg]] meetings are on Mondays at 6pm UTC by default every other week, but they can be moved :)
F: organising circle, started thinking about this 2 meetings ago.
[[flancian]] AI: to talk to the organizing circle about the mapping of user journeys to ways to engage with the coop.
In general, given current TWG capacity, our position is: we shouldn’t run anything we don’t have to run. So loomio/nextcloud: better to get it hosted, like we do currently (either through the main instance of loomio or mayfirst in the case of nextcloud)
On ways forward for Mastodon
Character limits vs moving to hometown
[[calix]] bumping character limits would up the maintenance cost; but the cost of switching to hometown would be high upfront.
[[flancian]] +1, unclear what would be better for TWG members of the future if this generation disappears. Maybe moving to hometown and reducing the maintenance cost upgrade-to-upgrade would be preferable. But then again hometown would eventually ‘fade away’.
[[calix]] fair but it might be preferable to assume there will be some TWG continuity; which is likely to be the case given the number of members.
[[Evan Boehs]] there’s the issue of Hometown not being updated in the last few months.
On the character limit: looking at the codebase, if the only thing we really want is the possibility of increasing the character limit, that seems to be just one file to update.
[[Calix]] we believe it’s two files — but still not a lot of files. The issue is that making customizations to the instance would require us to build our own docker images, which would increase the work that the TWG needs to do (even for an e.g. security update, of which Mastodon has quite a few).
[[Calix]] on the path to move to Hometown: it’d mean holding in the 4.2 branch but we could still update (apply security fixes).
[[Evan Boehs]] the docker arguments makes a lot of sense. One question about that: could we just ‘hot patch’ the image? Another: glitch soc could be an alternative?
[[flancian]] will bring it up with the [[CWG]] as there was recent discussion about [[meatspace]] gatherings; this seems related. I could also see the [[FWG]] being interested in having a list of projects that can be funded for different purposes, although the focus so far was closer to supporting digital-first efforts.
what is the twg’s opinion on the feasibility of patching mastodon to increase the value of StatusLengthValidator::MAX_CHARS
[[flancian]] if we apply a patch, or modify the instance in any way, the next step would be to set up alpha.social.coop. A good starter task for a new member ;)
next action: give it a stab / share if we get something working, maybe a python/requests program?
the spam might have been trying to ‘make a point’ this time
it seemed quite targeted/singled out certain users
we could have a mastodon release announcement bot in matrix?
maybe there’s a release bot for github CI?
single-sign-on
there’s interest in SSO in the organizing circle
F: great place for us to get a feel of what other working groups need. requests may naturally flow through org. circle, instead of needing full-graph connectivity between working groups
F: next steps? auth.social.coop working, open. test logging in as admin (deets in pass)
F: invite link could work well for new users. currently we send invite link (CWG) by email which isn’t really secure. could replace signup screen with hardcoded "reach out to admins". low-tech solution. CWG also brought up that it would be nice if some part of the wiki could only be edited by people in a working group.
E: the organizing circle was asking about the calendar, and how to restrict editing to some people.
Our position: we prefer [[trust but verify]]. Meaning: we’ll check ‘is in social coop’ programmatically; but everything else is up to group trust/human protocols.
F: attended first meeting, missed second, hoping to hit next one. carried question of running more services. want to get a sense from community about what folks are interested in having.
shared https://anagora.org/twg+2024 with them, suggestion to poll the community for which services they would like to see take shape
Andrew: this should be bidirectional, a shared discussion
More generally the interaction between working groups must be n-directional
Andrew is interested in the interface with the ‘real world’ of laws and finance
What is the scope of this? What should we call it?
F: "goalsetting" sounds good. but what about new people? "call for contributors" / "call for feedback". "open day"
C: really like "open day" / "open house"
C: Shoot for March deadline for poll, mid-late April for meeting?
F: align priorities in next month. come up with tasks. community call: these are the tasks up for grabs. make it concrete for people to see how they could contribute.
F: even though migration to SSO seems like a bit of would be cool to write docs on it to benefit the federation.
other services/more integrations?
this seems like it could be a good cooperative effort with [[coop cloud]]?
F: would love to run new services on [[coop cloud]]. examples of recipes where you can optionally compose containers. ideally would love to base mastodon SSO on keycloak. what else? people have been asking for other services. could do a poll?
C: other platforms like [[cloudron]][[sandstorm]] have a little more integration between apps but not much, would also be a great opportunity for documentation / collaboration. gaps: mastodon-to-archive, mastodon-to-other-instance, matrix on top of SSO.
F: [[beeper]] interactions — messaging as the next social platform. pidgin, libpurple, moa.party (need to pay Elon Tax™ tho — although could be an option for smaller communities).
F: anti-fedipact position is based on wanting to help users get out. corporation which interops is offering ways to defeat walled gardens. give users an option.
related communication on the instance w/[[calix]], [[ntnsndr]], [[flancian]] et al
[[meta]] it is quite a bit better than loomio for this kind of conversation…
C: TWG could work on importing Twitter / Facebook archives into mastodon, on behalf of social.coop.
F: Like the bidirectionality where it encourages companies to add interoperability. Also usability of logging in is better than e.g. Twitter Takeout.
Let’s try to do both? :D
F: Moderation and federation actions should be federated within activitypub, multi-software. Federation within groups not just within instances. Threads is the next iteration of scaling. Maybe reasonable to model as sub-networks; reasonable people, fascists — saying we won’t federate with fascists. A lot of the work of fediverse is banning fascists. That needs to be automated; that’s a lot of work.
C: this seems like something worth exploring; it seems like it would require support by threads.net and/or other large instances, like for example mastodon.social. Also have hope that we can achieve this on a cultural and technical level to make it easier to host smaller servers than large monolithic ones overall (vs current reality where it’s kind of the opposite).
F: If fediverse ends up being decentralised; improving the toolchain so people can mostly be on small instances, that would be the best outcome. We should hedge and try and do both. As long as there’s a motivation under capitalism to centralise it will continue to happen. In 5-10 years there will be new platforms, formats, which could again see capitalist entities trying to centralise for profit. Fediverse could be more resilient. Idea: write an addendum to Fedipact which says "this is how Threads could federate". FRom a point of view of the company, harder to push back on specific technical demands.
C: they certainly find it easy to push on ethical arguments; trying to get them on a technical argument might make sense.
F: Hedging, we can see which of the toolchains / approaches we invest most in, happy to start with 50:50. Personally wany to try the corporation negotiation approach. Would take any chance to push FB in the right direction. Also full respect to people who say the safest thing is to not federate for now. Happy to help on both fronts.
C: +1 — we talked about this back in September. Would suggest:
Plan A: we do a one-off date poll for a [[social coop tech working group goalsetting]] session. People commit to attending that; then we discuss further engagements there.
Plan B: we work on a roadmap ourselves and we announce the roadmap at the same time as the poll.
A: would like to do SSO sooner vs later, then fewer people will need to go through migration steps
E: Concerns about doing things just because we can (unless we’re tightening up things we’re already doing), e.g. bringing new services online — does someone other than us want them?
A: there has been interest in SSO, as well as some interest in Hometown
A: Hometown has been stuck on v4.0.x so it’s of some concern
C: another concern is that there isn’t a docker setup, so we would have to build our own, or move away from Docker at social.coop and lean more on Ansible
C: do we want to add something to our roadmap about the Organizing Circle, to help us to communicate with other working groups better? e.g. how we rotate our representation there, what we should tell them, what we should ask
C: SSO does seem like a priority, for users to be able edit wiki they need to ask for a SSO account
E: Agree, it’s a little better than what we had before, but not good that it’s still gate-kept by the extra signin. Is the question whether to use SSO for social.coop mastodon?
A: Yes. wiki already uses it. Loomio we can’t unless we selfhost. So just mastodon. Saw a similar migration done on another instance, was a bit of work.
A: we had good communication channels with our users via email about the change, and letting them know the change to the login
C: I’ve only set up SSO for hometown, not sure how to hide the login button but it is important
C: You can pay Loomio more money to get SSO working. Agreed communication would be key, we would frustrate and potentially lose users if we didn’t communicate properly.
C: We did talk about migrating passwords, but we established that we couldn’t because mastodon and keycloak use a different hashing algorithm
F: backups and restores are high priority still
F: we could extract email addresses from Mastodon, email them to create a new username and password, and give them a month, and then have a cutover. This would allow us to handwave migration.
C: if there’s an ordering to the roadmap we could have SSO first; I think it would be confusing if you could login to mastodon with either approach (local vs sso).
F: agree
A: we also did not have both, it was switched over on a particular day
F: we can generate a CSV of usernames and generate accounts for them, and then email them links to complete the setup
We should probably use a test instance to verify SSO/Keycloak is working with Mastodon.
C: might be an opportunity to better integrate Ansible and coop.cloud tooling (abra)?
C: switching to hometown will need additionally planning since downgrading would be difficult/tricky ; we would need to slowdown updates to social.coop to let it catch up
F: we should leave in the roadmap in case people want to discuss, but not a priority
C: probably should put behind the wiki and SSO; we also should prioritize organizing circle involvement
F: still is a question about Bonfire, but we can postpone discussion
C: I wanted to ask about blip to access, on Sunday Oct 1. I didn’t get around to seeing if datadog would be helpful to check what happened, also it’s not clear if we are getting alerts about availability. I could ssh to it at the time.
A: I did look at it and couldn’t find any evidence; the Datadog credentials are in the password store. We don’t monitor a bunch of things, and maybe we should talk about that. Should we include Datadog/Grafana plans as part of the roadmap?
C: at coop.cloud we were using uptimerobot for a bit; we are doing self-monitoring, uptime-kuma at the moment; even signing up for a free service like pingdom.
F: we should get it (monitoring) into the roadmap and make a request
[[flancian]] attends the first few meetings as representative if nobody else volunteers (by default) :)
we rotate as people want/can, [[Calix]] volunteered whenever time frees up :)
Issue of attendance
Low attendance to these meetings as of late
Common after summer breaks, but we want to make sure we keep critical mass going forward
Idea: start writing a roadmap document with possible next steps for the group, good opportunity to converge on plans and get them reviewed by other working groups and the social.coop community
Wiki next steps
We need to set up and test backup/restores
[[Flancian]] tested backups for the wiki, restores come next :)
first finding: abra app backup auth.social.coop worked as expected I think — it left me with a (quite small) auth_social_coop_db tarfile in ~/.abra/backups. this is nice :)
second finding: abra app backup wiki.social.coop says ‘no backup configs discovered for wiki.social.coop’ — maybe the recipe doesn’t support this yet?
Fl: Yes! Intended to close it down, so we don’t have spam issues. Now is a good time.
Ed: Did we want self-registration off?
Fl: Another member was surprised it was on. "What we want" is the right topic. How to integrate with registration form? How could you allow people to "sign in with X"?
Ak: Better in the long-run to go the other way - logging in with Keycloak into mastodon.
[[akshay]] ideally we’d run the instance in mastodon.social.coop and do "domain magic" to have accounts still be @social.coop, but that ship has sailed
maybe let’s try once we run bonfire or something else?
Maybe they’re reusing the API key or they’re making a lot of requests for the web interface?
And Mastodon really should handle this better, but it doesn’t — maybe all the other instances are backed by multiple servers and load-balance so the user takes way longer to hit this condition
Q: Is there any way to have per-user request logs in Mastodon?
A: no, there is not — but we could change mastodon_log_level in roles/social.coop/defaults/main.yml to debug (it’s info)
Next action (Flancian):
report back to the user, tell them what we found
ask optionally for an IP address so we can skip nginx logs to check how many requests the server is actually handling for them
I still need to test the latest changes in wiki-alpha; I will try to do that before the meeting tonight, or else we could do it live (tm) :)
if those work we’d be ready to set up wiki.social.coop using the same stack/steps. the only remaining step at that point would be to either special case the registration form path within wiki or (my preferred option) move the form to e.g. join.social.coop out of wiki.social.coop, which IMHO would make more sense?
I still need to test the latest changes in wiki-alpha; I will try to do that before the meeting tonight, or else we could do it live (tm) :)
if those work we’d be ready to set up wiki.social.coop using the same stack/steps. the only remaining step at that point would be to either special case the registration form path within wiki or (my preferred option) move the form to e.g. join.social.coop out of wiki.social.coop, which IMHO would make more sense?
Check that certbot is actually running in rhizome successfully if we didn’t do it back in February.
The certificate expires [[2023-04-17]] so there’s plenty of time, but certbot is not running — certbot.service expects it in /usr/bin/certbot but it’s installed in /usr/local/bin/certbot
The certificate expires [[2023-04-17]] so there’s plenty of time, but certbot is not running — certbot.service expects it in /usr/bin/certbot but it’s installed in /usr/local/bin/certbot
Port wiki-dev to our shiny new coop cloud mediawiki instance wiki-alpha see issue for details.
[[Flancian]] I discussed migration paths with the [[wiki builders]] chat and the easiest way forward is probably to declare maintenance for our wiki (including registrations?), bring it down and bring up mediawiki on wiki.social.coop during that window.
Eduardo: originally from Argentina (near Buenos Aires), now in Zürich
Akshay: originally from India, now in Berlin
Tod: based in Utah
Cleanup:
[runko] Run certbot commands to revoke the previous certificates.
Not needed, actually could be harmful as we copied the certificates.
Instead we should check that certbot is successfully renewing, moved to future agenda.
check that backups work
They don’t :)
Trying to fix it now, got all the way to setting up timer and fixing pg-dump-to-s3.service (prune was failing as we have a new server which means a new path in s3).
Open question is whether we need the assets:precompile step or, as the release notes seem to imply, we can skip because we’re using the prebuilt images?
We removed the precompile in the end.
Done!
Always run --check -vv --diff before going live
This time it prevented us from having another DEEPL outage because of a mismatched collection version in [[ansible galaxy]]
Issues we ran into
Had to run export PASSWORD_STORE_DIR=~/.password-store/social.coop (this depends on where you put your password store)
Had to run ansible-galaxy collection install -r requirements.yml in ansible directory
Tod: what about mediawiki migration?
Next steps
Make sure that [[coop cloud]] is running successfully in server [[hypha]]
We agreed on 2022-01-23 to schedule a maintenance window to test the database restore on Hypha (our new Hetzner server). If all goes well with the restore and testing we may decide to leave it running there.
includes certbot and certbox specific nginx config.
moved datadog password from ansible vault to pass store, lookup('passwordstore'...). we can use this for everything, instead of needing 2 systems
social.coop/defaults/main.yml is where a lot of the parameters for our configuration now are, including everything mastodon (database, s3 buckets, etc.)
replaced ‘docker-compose’ with ‘docker compose’ invocations, as the earlier is deprecated/removed in new versions.
added datadog logs parameters to docker-compose, but they aren’t used by default (require further configuration)
created an env file for postgres (to pass the password)
a certificate is generated first, that is needed so nginx can start; this runs once as it’s set up to check for existence of a file
whether to use secrets the way the wiki has them in the rhizome playbook, or the other way (which risks people using prod secrets in their test environments)
whether to use PASSWORD_STORE_DIR or include a social.coop/ prefix for every pass.
how to do the actual migration to rhizome once it’s set up.
[[calix]] would any of these changes affect the mastodon upgrade process?
[[akshay]] we’d need to automate some of the migration steps; currently docker compose config doesn’t include this.
when we restart mastodon, we restart everything — including elastic search, postgres. we need a more granular way to restart things.
We wouldn’t be ‘locked in’ based on the recipes — and the recipes are high quality and we could contribute back!
[[calix]] in autonomic we were using ansible for provisioning services and users and using coop cloud for app deployment and maintenance, but there is a project to move away from the abra cli even: [[wiki cafe]] ~ https://wiki.cafe
[[calix]] how much do we need to consider the "beach factor" (how many people does it take to go to the beach before you forget how to do things) when thinking about the approach. there’s been some turn over in TWG people, and knowledge transfer has proven difficult.
David: running an experiment with the abra CLI should be straightforward; biggest issue was how abra installs everything in a single machine.
Attending: Dan, David Vasandani, Akshay, Eduardo (flancian), Ed (edsu), LibreEquity, Calix (3wordchant)
Dan: upstate New York
(round of introductions)
Check ins
Dan: wrote a draft on Mastodon Verification for the publication they work with. [[rel equals me]].
Akshay: been away for >1mo, was in vacation (to India!) and travelling (now in Italy!). One more week of holiday left! May have time to dedicate to social.coop.
Ed: been kind of sick / with the flu. Otherwise going well and into vacation mode until EOY! Open to do coworking.
Nextcloud calendar for coworking?
LibreEquity: doing well, very interested in the Twitter -> Mastodon move. Read Nathan Schneider’s book in 2014-2015, followed him back then and had this in the back of our mind.
David: life is good, kids are healthy :) Getting warmer (in US). Diving into monitoring. Grafana, using docker, learnt a lot.
Calix: doing alright :) Neat people starting at autonomic.
Eduardo: entering production freeze, feeling nice about it :) also re-re-finishing (THIS TIME FOR REAL (tm)) [[agora chapter]].
Hacked around for the last few weeks. Last experience with monitoring/debugging/logging was a few years back. Main intent was to explore modern options in the monitoring/debugging/logging space.
Loki: tool for logging.
Taskfile is like a Makefile but in YAML.
Prometheus: Borgmon-like :) Gathers data.
Grafana: UI layer? Frontend for Prometheus.
docker-compose files for the whole stack. Will add this to a repo in git.coop.
statsd, fluentd.
Open question: homegrown docker-compose long time or [[coop cloud]]/[[abra]]? Is this a choice or are they complementary?
Akshay: use the same stack pretty much at work. You can configure loki to store logs somewhere — like an S3 bucket — (and that way you do not need extra indexing infra?).
David: we pipe things through fluentd to consolidate logs.
David: [[datadog]] can get expensive with one wrong logging statement :)
Flancian: is this the kind of stack that we could run in Rhizome now?
Akshay: would recommend we run monitoring in a separate server, because.
Ed: is the goal of this to be able to detect emergencies?
David: Grafana supports setting up alerting.
Akshay: Alert Manager is provided by Prometheus as well as an alternative.
Next steps?
Flancian: could we install this monitoring stack in [[hypha]]? ;)
Seemingly yes, seems like a good idea.
David: any concerns with network traffic? Would like to have someone else try it out, and then work on deploying it. We would need something installed at the system level and/or docker level (on runko)
David: Decide if we want SSO, and if we want a stand alone SSO.
David: [[keycloak]] might provide a way to have a separate SSO
Akshay: have some friends who migrated from Mastodon to Keycloak :)
Calix: would like to pick your brain sometime :) Were password resets required?
Akshay: no.
Calix: screensharing.
Issue with the sign in dialog in Hometown/Mastodon when two login providers are enabled.
We should go all-in, no dual login.
Flancian: question about the issue we’re trying to work around.
Patch is one line to add the email field in responses, could be mounted as a volume patch in docker-compose.
But it would increase maintenance.
Calix: note that dokuwiki doesn’t use this email field except for sending notifications, could we poll folks on preferences between disabling that vs. login system churn.
It might indeed make sense to patch dokuwiki instead of patching xMastodon, given that the earlier is less critical (and a newer service).
David: does this maybe depend on which other services we want to run?
Calix: yes, not only the number but also whether they use the email field the same way, or if they are based on username.
LibreEquity: is social.coop running an email server?
Flancian: nope, using webarchitechts forwarders set up on request
NEXT EPISODE:
Nextcloud access for other [[working groups]] (flancian)
Possible default policy: we batch create accounts for all WG members proactively and DM them their passwords? ideally over Matrix
Experiments with coopcloud.tech (protean)
Chat log dump follows:
Dan says:I like jitsi better than bbb
David Vasandani says:( * ^ *) ノシ
calix (they/them) says:dang this is a pleasantly large group! 🥰
me says:
https://doc.anagora.org/social-coop-tech-group?edit for notes 😃
Dan says:I’m on Firefox!
Dan says:(I use the Nightly build)
calix (they/them) says:it’s -9°C here today 🥶
Dan says:wow that is v cold!
calix (they/them) says:👍
David Vasandani says:👍
Dan says:a previous Mastodon article I helped out with
https://themarkup.org/the-breakdown/2022/11/21/we-joined-mastodon-heres-what-we-learned-about-privacy-and-security
calix (they/them) says:
https://git.coopcloud.tech/coop-cloud/monitoring
calix (they/them) says:
https://git.coopcloud.tech/coop-cloud/monitoring-lite
calix (they/them) says:think it’s the same stack?
David Vasandani says:@calix it does look to be the same!
David Vasandani says:I believe the only difference is I put fluentbit between the service and loki.
calix (they/them) says:👍
calix (they/them) says:👏
me says:
rhizome.social.coop
edsu says:👏
Dan says:unrelated but I have some affiliation with an NYC org
rhizome.org
LibreEquity says:RAID0?
edsu says:brb
edsu says:Here’s an issue where I wrote down some notes on the dokuwiki/mastodon sso experiment:
https://git.coop/social.coop/tech/operations/-/issues/53
Dan says:"hey are you talking about mastodon?" –my 3yo
calix (they/them) says:🤣
edsu says:lol
LibreEquity says:👍
edsu says:😀
LibreEquity says:😀
David Vasandani says:Great point!
edsu says:I disabled that dual login in the dokuwiki — agree
David Vasandani says:Thank you for that call out.
edsu says:💯
calix (they/them) says:oh and for completeness we hid the username/password fields in CSS 🕵
calix (they/them) says:
https://github.com/cosmocode/dokuwiki-plugin-oauthdoorkeeper/compare/master…edsu:dokuwiki-plugin-oauthdoorkeeper:mastodon
David Vasandani says:Never be sorry!
David Vasandani says:Not everyone has an alias.
David Vasandani says:It by request.
running bonfire community; some upstream issues w/ bonfire (e.g. mail + mailgun)
good support in bonfire chat
goal is to find something that has more governance built into the same plaform; we have difficulty getting everyone on all the platforms (loomio, matrix, git, etc) would be good to have governance folded in.
there doesn’t appear to be voting built in, but since it is modular it should support writing an extension
if anybody wants to take an account feel free ; just look for confirmation link in your spam
@davidvasandani just created an account but didn’t get an email.
@protean shared and link and @davidvasandani was able to access.
participation rate is like 15%-30% of people who are loomio, and people who are on loomio is 50% of active mastodon
did a poll for translation feature, some people said thank you for doing it that way
jotaemmei: sometimes I don’t vote because I see that something is going to pass
if loomio is asking people to jump through too many hoops we should re-evaluate
how do we make sure we accomodate people’s attentional needs
this is a larger issue with the larger confederation of co-ops, but it is an issue even internal with social.coop
some people have signed up to do bridging work, but it’s unclear how to manage it going forward
[[flancian]] should we reboot runko at some point? :)
[ ] Yes we should! let’s schedule that for Friday 2022-12-10
- follow up on issues in [bug tracker](https://git.coop/social.coop/tech/operations/-/issues)
- [#52](https://git.coop/social.coop/tech/operations/-/issues/52) - Datadog Agent
- Do we have an account? Yes, info is in the pass.
- Plan size?
- Access?
- protean: there was no systemd service to start datadog,
- got it working again, and added logging
- not connecting to redis and elasticsearch because it’s not running in those docker containers
- @davidvasandani can help configure if we’d like to mornitor.
- we need to decide whether we want to keep using it, someone mentioned prometheus?
- @davidvasandani is testing Prom, Grafana, and Loki locally for monitoring Mastodon.
- We can discuss the pros and cons of build vs buy and what our objectives are without getting too technical.
- [#56](https://git.coop/social.coop/tech/operations/-/issues/56) - Experiments Server
- Currently setup on Hetzner (hypha)
- Is there interest in full E2E automation and testing of new versions and restoring from backups?
- Does the group use or have experience with Terraform?
- If there's interest @davidvasandani would like to run point on creating the automation and running some pairing sessions.
- [#53](https://git.coop/social.coop/tech/operations/-/issues/53) - Setup Dokuwiki
- Test setup on hypha? (Hetzner experiments server)
- Has someone picked up this ticket? Can @davidvasandani pick this up? (might want to touch base with @flancian)
new server!
we have the go ahead to use up to $250 per month
should select a server and put to a quick loomio checkin/vote
- protean: opened the log retention issue on loomio
- https://www.loomio.com/d/kXMdf0ZW/privacy-policy-log-retention-and-anonymization
- jota: what do we have now for a privacy policy?
- protean: it links to the code of conduct at the moment
- jota: there are questions of jurisdiction that I was hoping the legal working group could help address
- protean: i want the lawyer people to provide guidance here
- protean: i posted a draft text about what we are currently doing
- protean: there is .env configuration to scrub login information we could think about doing
- david: would there be interest in moving to using something like terraform for provisioning the Hetzner machines? that way we could bring up experiments and tear them down easily.
- there was general agreement that this sounded like a worthwhile thing to look at
- david is going to set up a time for the TWG to discuss how we want to evolve our infrastructure code
- schedule and announce the next meeting time
- moving to bi-weekly meetings, that meet a bit later for Bo, and hopefully are still doable by flancian?
- re: AWS/Azure 501c3 grants
- social.coop is not a 501c3 and the overhead of creating one or using someone elses didn’t seem feasible
these are not public to the internet — they are just on the ‘external’ docker network
[[david vasandani]] should we set up a separate meeting to talk about tech roadmap level things, or use this meeting?
this meeting should be a good forum.
[[ian]] also a lot of this discussion is advancing well in the matrix chat rooms.
[[edsu]] a lot of these are bound up with what social.coop (the larger group) wants to do, we should take care not to over-focus on the tech aspects only
[[david]] thinking mostly of incremental improvements to our current stack
[[david vasandani]] what’s the advantage of these experiments w.r.t. reliability? also, two servers might not affect reliability in the positive direction.
(…)
[[ian]] on the parallels between the fediverse now and email 20+ years ago. Things got hard with running your own email because of spam. There’s no reason to believe that this won’t happen with the fediverse, if we just let it go. People will start to demand more moderation policies, if people don’t want to be on your instance because people aren’t federating with you, there will be re-centralization.
on Ian’s experiment: if we’re going to combat the previous, we need some way of maintaining the distributed aspects of the network. This requires instances to agree on codes of conduct, standards for handling signups.
aside: could social.coop scale up to ~50k people with loomio?
-> we need an instance that can self-govern.
[[dphiffer]] proposed to union local that they run an instance.
but what does it mean for one coop to be a member of another coop?
[[ian]] it might or might not make sense depending on how the instances operate; for example higher level coops could vote on higher level rules on how associated instances federate.
lack of good tools for content moderation is a problem.
the fediverse currently has only hashtags to have distributed decision making on e.g. blocking
a coalition of federated instances could do better; they could agree for example
[[flancian]] we could also imagine extending the ActivityPub protocol to support sharing this information between instances; this is something the fediverse needs to solve if it’s going to scale
@Flancian: Happy after weekend playing with containers. Way fewer (late) meetings because of thanksgiving.
@Edsu: starting week happily.
@Protean: working on not taking down the site today ;)
@3wordchant / @3wc / Calix: doing well, happy to have an individual account in social.coop now :) ran in the rain yesterday (pleasantly) and have had an interesting day so far (ethical wise).
sudo touch /opt/social.coop/var/www/maintenance_mode_on
sudo docker-compose stop web streaming sidekiq sidekiq-scheduler sidekiq-default-q sidekiq-push-q sidekiq-pull-q es
sudo systemctl start pg-dump-to-s3.service
sudo docker-compose exec db vacuumdb -U postgres --all --full --analyze --verbose
sudo systemctl start pg-dump-to-s3.service
# validate that the backup is in the bucket and that we are able to retrieve it, otherwise move manually to [[hypha]]
sudo docker-compose exec db pg_dumpall -U postgres > ../../var/lib/postgresql/9.6.backup
sudo docker-compose down --remove-orphans
sudo git checkout 3_5_upgrade
sudo docker-compose up -d db
sudo cat ../../var/lib/postgresql/9.6.backup | docker-compose exec -T db psql -U postgres
sudo docker-compose logs -t db
sudo docker-compose run --rm -e SKIP_POST_DEPLOYMENT_MIGRATIONS=true web rails db:migrate
sudo docker-compose down --remove-orphans
sudo docker-compose up --scale sidekiq-pull-q=4 --scale sidekiq-default-q=12 --scale sidekiq-push-q=4 -d
sudo docker-compose run --rm web bin/tootctl cache clear
sudo docker-compose run --rm web rails db:migrate
sudo docker-compose down --remove-orphans
sudo docker-compose up --scale sidekiq-pull-q=4 --scale sidekiq-default-q=12 --scale sidekiq-push-q=4 -d
Done
==ROLLBACK==
sudo docker-compose down --remove-orphans
git checkout rebuild
sudo docker-compose up --scale sidekiq-pull-q=4 --scale sidekiq-default-q=12 --scale sidekiq-push-q=4 -d
option one: consolidate (move) twg.wiki (tech/operations repo), cwg.wiki (community/operations) into general.wiki, which gets processed and deployed from ansible?
option two: change [[ansible playbook]] to also check out the twg and cwg wikis and somehow integrate them at deployment step
pros/cons of these approaches
do we want to keep anything in the wiki secret?
[[ian]] there might be a grey area for things like management interfaces (URLs), but by and large the right default option is public
things that keep your logs clean
some documentation lives with the ansible repo as well
any pros to option two?
cons: makes linking between repos more difficult, [[edsu]]: it’s nice to think of it as one knowledge base
[[operations]] source of truth for our tech group work
nine repositories currently, counting three wiki
[[flancian]]: I’d like to designate one repository as ‘clone this first’ and add a ./bootstrap-local.sh script which clones all social.coop repos to begin with.
[[flancian]]: what’s the best repo for that? is it [[ansible]] because of how we want to use automation for this, in particular for ./bootstrap-prod.sh?
flancian: originally thought tech-operations but maybe ansible?
[[protean]] ~ [[ian]]: as he was doing the documentation revamp he leaned towards not putting everything in ansible
the setup we have may be partly influenced by the lack of group wikis in gitlab at the time of design
question: when you want to use the ansible repo, the way it’s intended to be used:
you have your ssh key in your local machine that you’re working on
you clone the ansible repo
you run the playbook locally and that connects and alters the server
we could make a playbook for things currently in the sauce
[[ian]] if we want to go to the server and clone something there, that workflow (which comes more natural to some) just doesn’t fit the above a lot
unclear if there is a big advantage of moving [[docker compose]] stuff to [[ansible]], it’s mostly one file
[[ansible repo]] should indeed install docker, docker compose though
[[flancian]] ansible did feel like a learning curve when starting
[[edsu]] should we use [[ansible]] to do upgrades? maybe we could tackle [[mastodon upgrade]] this way and learn from that?
[[ian]] even at work, ansible manages the stages from bare metal to deployment ready.
backup restores as a prereq for mastodon upgrade process? or is that too conservative?
second server for testing backup/restore
[[flancian]]: not comfortable doing upgrade without having done the restore; at some point we are going to have a problem and it will take us a few days to get back up; how was this done previously?
[[protean]]: when we moved from [[trunk]] (digitalocean vps) to [[runko]] (hetzner colocated) we had two servers
might be able to either
A. do a test restore in a droplet
B. run two compose instances on [[runko]], on different ports, there is a risk if you mess up.
📚 Node [[social coop tech group]] exact match
📓 social coop tech group.md by @flancian ️🔗 ✍️
$ tootctl accounts create <user> --reattach --force --email=<email>docker exec -it docker-web-1 /bin/bashin [[rhizome]]📓 social-coop-tech-group.md by @anonymous@doc.anagora.org ✍️
2025-03-10
[[Andrew]], [[Calix]], [[Dan]], [[Flancian]].
2025-02-10
[[Dan]], [[Edsu]], [[Flacian]].
2025-02-03
[[Dan]], [[Flacian]], [[Calix]] https://git.coop/social.coop/tech/ansible/-/merge_requests/51
2025-01-20
[[Dan]], [[Flacian]], [[Calix]]
https://agriculturaljusticeproject.org/toolkit/resources/relations/soulfire-courageous-conversations/ https://srinathramakrihttps://seedsforchange.org.uk/downloads/conflictbooklet.pdf shnan.wordpress.com/wp-content/uploads/2016/07/non-violent-communication-summary.pdf
Timeline
March 21, 2024: Meta announces that Threads ActivityPub integration is now available in the United States, Canada and Japan.
Jan 8th 2025:
flancian suggests a vote would be needed for taking action and "I’d recommend interested people ideally get together to agree on the phasing and then start one"
dnlbrnds and 3wc discuss co-drafting a proposal
Timeline highlights / overview
F: Feeling very invested in things going into the situation. Suggested vote on the 8th, saw people worrying. 9th read more reaction, started changing my mind from "limit" based on lack of practical impact, to "suspend" based on emotional impact. 10th felt like the issue escalated. My POV: asked to wait. Felt like people were not OK with that. Suggested foolishowl create a thread. Didn’t see pushback against calling a vote. Started a vote 1.5 hours after sending draft. Felt like reaction that took place after was not as constructive as it could have been. Felt responsibility as CWG member to start the vote. Did not feel in a good position to manage the conflict. Assumptions about personal motivations. Led to feeling disappointed.
C: Felt disappointed as well when I saw the vote framed in this way. Felt that future at coop was uncertain. Suspicious about the shortness of the draft review. Felt unseen/heard.
Next actions
2024-01-13
2024-11-25
sudo docker exec -it docker-web-1 tootctl statuses remove --days 25002024-11-04
2024-10-07
2024-10-07
2024-09-23
2 years, social.coop for4. Joined the CWG, and take some shifts, and also involved in the organizing circle. Super into cooperatives, and work in big-tech as part of daily life (e.g. this is meeting no. 8 today). Also from Argentina.2024-08-26
Here: [[Kathe T-B]] [[Eduardo Mercovich]] [[Flancian]] [[@andrewe]]
Gandhi access for domain:
Two Edus remain
social.coop
recomendaciones
2024-08-06
cd ~/.password-store/social.coop && passpasssecret:pass show social.coop/path/to/secretwikiis actually for join.social.cooprhizome.playbook.ymlconfigures the live server that runs social.coophypha.playbook.ymlis for an experimental serverserverrole has public keys for login access2024-07-29
2024-07-15
2024-07-01
2024-06-01
Here: [[flancian]] trying to bump to the latest security patch :)
2024-05-20
Here: [[Calix]] [[flancian]]
yarn buildthat might add a minute or so to every container start⥅ [[https://doc.anagora.org/uploads/upload_d18b0a750f7762179948fcaa71edeb63.png]]
2024-05-06 — Long time no see! (at least with notes?) :)
Here: [[flancian]], [[dphiffer]], [[evan boehs]]?
/bin/sh -c "ls && sed -i -e 's/500/1000/g' app/javascript/mastodon/features/compose/components/compose_form.jsx && sed -i -e 's/500/1000/g' app/validators/status_length_validator.rb && RAILS_ENV=production rails assets:precompile && /start.sh".2024-04-13 — Tech Working Group Open House!
Here: [[Calix]], [[flancian]], Melissa (ansate), Jamie, Caitlin
2024-03-25
Here: [[Calix]] [[flancian]]
2024-02-27
Here: [[Calix]] [[edsu]] [[flancian]]
tootctlhave something?pass)2024-02-13
Here: [[flancian]] [[Calix]] [[andrewe]]
2023-12-18 .. [[2024-01-01]] .. [[2024-01-15]]
Here: [[flancian]] [[Calix]] :)
Wish you all a great holiday season! Some topics:
2023-12-04
Here: [[edsu]], [[Flancian]]
2023-10-09
Here: [[Akshay]], [[edsu]]I’ll b, [[Calix]], [[Flancian]]
[[2023-09-26]]
Here: [[Akshay]] [[Flancian]] [[Calix]]
Mastodon upgrade party!
[[2023-09-25]]
Here: [[Akshay]] [[Flancian]]
[[2023-09-11]]
Here: [[Flancian]]
[[2023-08-28]]
Here: [[Calix]] [[Flancian]]
[[2023-08-14]]
Here: [[Flancian]]
abra app backup.abra app backupworks for wiki.social.coop \o/[[2023-07-17]]
Here: [[Akshay]] [[Flancian]] [[Calix]]
[[2023-07-05]]
Here: [[Akshay]] [[Ed Summers]] [[Flancian]]
[[2023-06-19]]
Here: Akshay, Flancian, 3wc/calix
[[2023-06-05]]
Here: edsu, Akshay, Flancian, (a late Calix)
[[2023-05-22]]
Here: Akshay, Eduardo, Ed …
[[2023-05-15]]
Here: Akshay, Eduardo, …
[[2023-05-08]]
Here: Eduardo, …
[[2023-03-27]]
Here: Eduardo, Akskhay, Calix
[[2023-03-13]]
Here: Eduardo, Akskhay, Calix
[[2023-02-27]]
[[2023-02-13]]
--check -vv --diffbefore going liveexport PASSWORD_STORE_DIR=~/.password-store/social.coop(this depends on where you put your password store)ansible-galaxy collection install -r requirements.ymlin ansible directoryLocalSettings.phpfile)[[2023-01-30]]
We agreed on 2022-01-23 to schedule a maintenance window to test the database restore on Hypha (our new Hetzner server). If all goes well with the restore and testing we may decide to leave it running there.
This checklist was copied from a comment that Akshay put in a GitCoop issue.
Arecord forsocial.coopto something like 300.systemctl start pg-dump-to-s3.serviceArecord forsocial.coopto point to rhizome, make sure TTL is small to help us revert if needed.Cleanup:
log
[[2023-01-23]]
sneakers the ratis working on this[[2023-01-09]]
lookup('passwordstore'...). we can use this for everything, instead of needing 2 systemsgit clone --recurse-submodules)[[2022-12-19]]
addgroup wheel[[2022-12-05]]
@davidvasandani just created an account but didn’t get an email.[ ] Yes we should! let’s schedule that for Friday 2022-12-10 - follow up on issues in [bug tracker](https://git.coop/social.coop/tech/operations/-/issues) - [#52](https://git.coop/social.coop/tech/operations/-/issues/52) - Datadog Agent - Do we have an account? Yes, info is in the pass. - Plan size? - Access? - protean: there was no systemd service to start datadog, - got it working again, and added logging - not connecting to redis and elasticsearch because it’s not running in those docker containers - @davidvasandani can help configure if we’d like to mornitor.
we have the go ahead to use up to $250 per month
should select a server and put to a quick loomio checkin/vote - protean: opened the log retention issue on loomio - https://www.loomio.com/d/kXMdf0ZW/privacy-policy-log-retention-and-anonymization - jota: what do we have now for a privacy policy? - protean: it links to the code of conduct at the moment - jota: there are questions of jurisdiction that I was hoping the legal working group could help address - protean: i want the lawyer people to provide guidance here - protean: i posted a draft text about what we are currently doing - protean: there is .env configuration to scrub login information we could think about doing - david: would there be interest in moving to using something like terraform for provisioning the Hetzner machines? that way we could bring up experiments and tear them down easily. - there was general agreement that this sounded like a worthwhile thing to look at - david is going to set up a time for the TWG to discuss how we want to evolve our infrastructure code - schedule and announce the next meeting time - moving to bi-weekly meetings, that meet a bit later for Bo, and hopefully are still doable by flancian? - re: AWS/Azure 501c3 grants - social.coop is not a 501c3 and the overhead of creating one or using someone elses didn’t seem feasible
[[2022-11-28]]
[[2022-11-21 18:00 UTC]]
Location: https://socialcoop.meet.coop/dav-y3e-c21-sgv
Attending: 3wc, edsu, protean, flancian
Mastodon upgrade!
Check in
Meta
Open questions
Steps that @protean put together for the upgrade
[[2022-11-14 19:00 UTC]]
abrawhich uses [[docker swarm]] to do [[zero downtime deploy]]Several past instances had notes only on [[loomio]]
2022-05-27
⥅ [[https://doc.anagora.org/uploads/upload_ad643dbf540e732ff8232559bb9973dd.png]]
2022-05-20
📓 social coop tech group.md by @agora@botsin.space
📓 social coop tech group.md by @an_agora@twitter.com
📓 social coop tech group.md by @anagora@matrix.org
📓 social-coop-tech-group.md by @anagora@matrix.org
📓 social coop tech group.md by @flancian@social.coop
Loading pushes...
Rendering context...