Wikitech-l October 2017

53 participants
70 discussions

How to make commits from mirrored Wikimedia repos show up on your GitHub profile
by Bartosz Dziewoński 27 Jun '24

27 Jun '24

I know it has been annoying a couple of people other than me, so now that I've learned how to make it work I'll share the knowledge here. tl;dr: Star the repositories. No, seriously. (And yes, you need to star each extension repo separately.) (Is there a place on mw.org to put this tidbit on?) ------- Forwarded message ------- From: "Brian Levine" <support(a)github.com> (GitHub Staff) To: matma.rex(a)gmail.com Cc: Subject: Re: Commits in mirrored repositories not showing up on my profile Date: Tue, 09 Jul 2013 06:47:19 +0200 Hi Bartosz In order to link your commits to your GitHub account, you need to have some association with the repository other than authoring the commit. Usually, having push access gives you that connection. In this case, you don't have push permission, so we don't link you to the commit. The easy solution here is for you to star the repository. If you star it - along with the other repositories that are giving you this problem - we'll see that you're connected to the repository and you'll get contribution credit for those commits. Cheers Brian -- Matma Rex

3 3

Research FAQ gets a facelift
by Dario Taraborelli 25 Jun '24

25 Jun '24

We just released a new version of Research:FAQ on Meta [1], significantly expanded and updated, to make our processes at WMF more transparent and to meet an explicit FDC request to clarify the role and responsibilities of individual teams involved in research across the organization. The previous version – written from the perspective of the (now inactive) Research:Committee, and mostly obsolete since the release of WMF's open access policy [2] – can still be found here [3]. Comments and bold edits to the new version of the document are welcome. For any question or concern, you can drop me a line or ping my username on-wiki. Thanks, Dario [1] https://meta.wikimedia.org/wiki/Research:FAQ [2] https://wikimediafoundation.org/wiki/Open_access_policy [3] https://meta.wikimedia.org/w/index.php?title=Research:FAQ&oldid=15176953 *Dario Taraborelli *Head of Research, Wikimedia Foundation wikimediafoundation.org • nitens.org • @readermeter <http://twitter.com/readermeter>

2 1

bluejeans
by Jeremy Baron 22 Feb '23

22 Feb '23

Hi, On Tue, Mar 1, 2016 at 3:36 PM, David Strine <dstrine(a)wikimedia.org> wrote: > We will be holding this brownbag in 25 minutes. The Bluejeans link has > changed: > > https://bluejeans.com/396234560 I'm not familiar with bluejeans and maybe have missed a transition because I wasn't paying enough attention. is this some kind of experiment? have all meetings transitioned to this service? anyway, my immediate question at the moment is how do you join without sharing your microphone and camera? am I correct thinking that this is an entirely proprietary stack that's neither gratis nor libre and has no on-premise (not cloud) hosting option? are we paying for this? -Jeremy

9 16

New DB_REPLICA constant; DB_SLAVE deprecated
by Aaron Schulz 04 Jul '20

04 Jul '20

As of 950cf6016c, the mediawiki/core repo was updated to use DB_REPLICA instead of DB_SLAVE, with the old constant left as an alias. This is part of a string of commits that cleaned up the mixed use of "replica" and "slave" by sticking to the former. Extensions have not been mass converted. Please use the new constant in any new code. The word "replica" is a bit more indicative of a broader range of DB setups*, is used by a range of large companies**, and is more neutral in connotations. Drupal and Django made similar updates (even replacing the word "master"): * https://www.drupal.org/node/2275877 * https://github.com/django/django/pull/2692/files & https://github.com/django/django/commit/beec05686ccc3bee8461f9a5a02c607a023… I don't plan on doing anything to DB_MASTER, since it seems fine by itself, like "master copy", "master tape" or "master key". This is analogous to a master RDBMs database. Even multi-master RDBMs systems tend to have a stronger consistency than classic RDBMs slave servers, and present themselves as one logical "master" or "authoritative" copy. Even in it's personified form, a "master" database can readily be thought of as analogous to "controller", "governer", "ruler", lead "officer", or such.** * clusters using two-phase commit, galera using certification-based replication, multi-master circular replication, ect... ** https://en.wikipedia.org/wiki/Master/slave_(technology)#Appropriateness_of_… *** http://www.merriam-webster.com/dictionary/master?utm_campaign=sd&utm_medium… -- -Aaron

4 3

Roadmap for CX?
by Strainu 06 Sep '18

06 Sep '18

Following the recent outage, we've had a new series of complaints about the lack of improvements in CX, especially related to server-side activities like saving/publishing pages. Now, I know the team is involved in a long-term effort to merge the editor with the VE, but is there an end in sight for that effort? Can I tell people who ask "look, 6 more months then we'll have a much better translation tool"? Is there a publicly available roadmap for this project and more generally, for CX? Thanks, Strainu

5 8

How to customize Preferences
by Sam Wilson 19 Dec '17

19 Dec '17

Hi all, (Reposting https://www.mediawiki.org/wiki/Topic:Tyvfh19mba4pway9 here to garner more input.) I'm working on Extension:GlobalPreferences and trying to figure out how best to do things with all preferences, after they've been defined (in order to show various extra Preferences-form bits and pieces that depend on knowing about all preferences). At the moment, we're using $wgExtensionFunctions and hacking the $wgHooks global to add a new callback at the end of $wgHooks['GetPreferences']. One idea is to add a new MediaWiki service called 'PreferencesFactory', that can be used to retrieve a new Preferences object. Extensions would then be able to use the MediaWikiServices hook to redefine the PreferencesFactory (with MediaWikiServices::redefineService()). Of course, only one extension would be able to do that (which maybe is a bit odd). Apart from being able to override the Preferences class, a service for this would also mean the Preferences class could be refactored (gradually?) to not be such a collection of static methods. The proposed patch is: https://gerrit.wikimedia.org/r/#/c/374451/ I'd love to hear anyone's ideas about this, including completely different and better ways to do things. :-) Another idea is to add a new hook, after GetPreferences. This wouldn't be as flexible as the PreferencesFactory idea, but is a lot simpler. Thanks, Sam.

1 1

Can we drop revision hashes (rev_sha1)?
by Daniel Kinzler 07 Dec '17

07 Dec '17

Hi all! I'm working on the database schema for Multi-Content-Revisions (MCR) <https://www.mediawiki.org/wiki/Multi-Content_Revisions/Database_Schema> and I'd like to get rid of the rev_sha1 field: Maintaining revision hashes (the rev_sha1 field) is expensive, and becomes more expensive with MCR. With multiple content objects per revision, we need to track the hash for each slot, and then re-calculate the sha1 for each revision. That's expensive especially in terms of bytes-per-database-row, which impacts query performance. So, what do we need the rev_sha1 field for? As far as I know, nothing in core uses it, and I'm not aware of any extension using it either. It seems to be used primarily in offline analysis for detecting (manual) reverts by looking for revisions with the same hash. Is that reason enough for dragging all the hashes around the database with every revision update? Or can we just compute the hashes on the fly for the offline analysis? Computing hashes is slow since the content needs to be loaded first, but it would only have to be done for pairs of revisions of the same page with the same size, which should be a pretty good optimization. Also, I believe Roan is currently looking for a better mechanism for tracking all kinds of reverts directly. So, can we drop rev_sha1? -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

16 29

Proposal regarding the handling of imported usernames
by Brad Jorsch (Anomie) 06 Dec '17

06 Dec '17

Handling of usernames in imported edits in MediaWiki has long been weird (T9240[1] was filed in 2006!). If the local user doesn't exist, we get a strange row in the revision table where rev_user_text refers to a valid name while rev_user is 0 which typically indicates an IP edit. Someone can later create the name, but rev_user remains 0, so depending on which field a tool looks at the revision may or may not be considered to actually belong to the newly-created user. If the local user does exist when the import is done, the edit is attributed to that user regardless of whether it's actually the same user. See T179246[2] for an example where imported edits got attributed to the wrong account in pre-SUL times. In Gerrit change 386625[3] I propose to change that. - If revisions are imported using the "Upload XML data" method, it will be required to fill in a new field to indicate the source of the edits, which is intended to be interpreted as an interwiki prefix. - If revisions are imported using the."Import from another wiki" method, the specified source wiki will be used as the source. - During the import, any usernames that don't exist locally (and can't be auto-created via CentralAuth[4]) will be imported as an otherwise-invalid name, e.g. an edit by User:Example from source 'en' would be imported as "en>Example".[5] - There will be a checkbox on Special:Import to specify whether the same should be done for usernames that do exist locally (or can be created) or whether those edits should be attributed to the existing/autocreated local user. - On history pages, log pages, and the like, these usernames will be displayed as interwiki links, much as might be generated by wikitext like " [[:en:User:Example|en>Example]]". No parenthesized 'tool' links (talk, block, and so on) will be generated for these rows. - On WMF wikis, we'll run a maintenance script to clean up the existing rows with valid usernames and rev_user = 0. The current plan there is to attribute these edits to existing SUL users where possible and to prefix them with a generic prefix otherwise, but we could as easily prefix them all. - Unfortunately it's impossible to retroactively determine the actual source of old imports automatically or to automatically do anything about imports that were misattributed to a different local user in pre-SUL times (e.g. T179246[2]). - The same will be done for CentralAuth's global suppression blocks. In this case, on WMF wikis we can safely point them all at Meta. If you have comments on this proposal, please reply here or on https://gerrit.wikimedia.org/r/#/c/386625/. Background: The upcoming actor table changes[6] require some change to the handling of these imported names because we can't have separate attribution to "Example as a non-registered user" and "Example as a registered user" with the new schema. The options we've identified are: 1. This proposal, or something much like it. 2. All the existing rows with rev_user = 0 would have to be attributed to the existing local user (if any), and in the future when a new user is created any existing edits attributed to that name will be automatically attributed to that new account. 3. All the existing rows with rev_user = 0 and an existing local user would have to be re-attributed to different *valid* usernames, probably randomly-generated in some manner, and in the future when a new user is created any existing edits for that name would have to be similarly re-attributed. 4. Like #2, except the creation (including SUL auto-creation) of the same-named account would not be allowed. Thus, an import before the local name exists would forever block that name from being used for an actual local account. 5. Some less consistent combination of the "all the existing rows" and "when a new user is created" options from #2–4. Of these options, this proposal seems like the best one. [1]: https://phabricator.wikimedia.org/T9240 [2]: https://phabricator.wikimedia.org/T179246 [3]: https://gerrit.wikimedia.org/r/#/c/386625/ [4]: https://phabricator.wikimedia.org/T111605 [5]: ">" was chosen rather than the more typical ":" because the former is already invalid in all usernames (and page titles). While a colon is *now* disallowed in new usernames, existing names created before that restriction was added can continue to be used (and there are over 12000 such usernames in WMF's SUL) and we decided it'd be better not to suddenly break them. [6]: https://phabricator.wikimedia.org/T167246 -- Brad Jorsch (Anomie) Senior Software Engineer Wikimedia Foundation

3 5

Important news about the November dumps run!
by Ariel Glenn WMF 07 Nov '17

07 Nov '17

As was previously announced on the xmldatadumps-l list, the sql/xml dumps generated twice a month will be written to an internal server, starting with the November run. This is in part to reduce load on the web/rsync/nfs server which has been doing this work also until now. We want separation of roles for some other reasons too. Because I want to get this right, and there are a lot of moving parts, and I don't want to rsync all the prefetch data over to these boxes again next month after cancelling the move: ******** If needed, the November full run will be delayed for a few days. If the November full run takes too long, the partial run, usually starting on the 20th of the month, will not take place. ********* Additionally, as described in an earlier email on the xmldatadumps-l list: ********* files will show up on the web server/rsync server with a substantial delay. Initially this may be a day or more. This includes index.html and other status files. ********* You can keep track of developments here: https://phabricator.wikimedia.org/T178893 If you know folks not on the lists in the recipients field for this email, please forward it to them and suggest that they subscribe to this list. Thanks, Ariel

2 4

Upcoming holidays/events and their impact on the deployment schedule
by Greg Grossmeier 06 Nov '17

06 Nov '17

Hello all, It's that time of year again where the density of holidays increases and at the same time our plans for fundraising also increase. Per our usual practice we will be not doing deployments at various points in the next few months. Here's the full outline: Reminder for all on we did last year: * No MW train the week of Thanksgiving (but SWAT deploys were open for high-priority things). * No deploys (at all) the last two weeks of December. People were happy with this. * The first week of January was normal (minus Monday being Jan 2nd, our observed New Year's Day holiday) deployment wise. * The second week was weird due to Dev Summit/All Hands: No MediaWiki train, only SWATs as needed on Mon/Tues/Wed. No SWATs/deploys during All Hands. I imagine we'll do similarly. In that case (looking at the calendar....) * No MW train the week of Thanksgiving (Nov 20th), SWATs open for high priority things * No deploys weeks of Dec 18th and 25th (last two weeks) * Normal week week of Jan 1st (minus no deploys that Monday) * The Dev Summit and WMF All Hands is the week of January 22nd, so that will be a "No Train but SWATs OK on Mon/Tues/Wed" week. * The following week (week of January 29th) the Release Engineering team will be on an offsite, so a week of "No Train, but SWATs and service deploys OK". This is now on-wiki at: https://wikitech.wikimedia.org/wiki/Deployments#Upcoming Best, Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l October 2017