Lerna looks interesting but as far as I can see we'd need to commit to it for all packages in our "monorepo" (and it's really not clear to me how that would work with submodules like termbox).

Moving to camp as we closed the hike!

Change 572828 had a related patch set uploaded (by Tarrow; owner: Tarrow):
[mediawiki/extensions/Wikibase@master] TR: Add as dev dependency of Wikibase

https://gerrit.wikimedia.org/r/572828

Change 573235 had a related patch set uploaded (by Tarrow; owner: Tarrow):
[integration/config@master] Add daily job to run npm audit on Wikibase

https://gerrit.wikimedia.org/r/573235

Assigning to myself to make the CI change happen!

Change 573235 merged by jenkins-bot:
[integration/config@master] Add daily job to run npm audit on Wikibase

https://gerrit.wikimedia.org/r/573235

Job merged and running.

See https://integration.wikimedia.org/ci/job/wikibase-daily-npm-audit-daily-node10-npmaudit-docker/7/console

Currently email notifications are turned off pending @Tarrow review of this and getting it green :)

Additional merged gerrit config patches:

• https://gerrit.wikimedia.org/r/#/c/582798/
• https://gerrit.wikimedia.org/r/#/c/integration/config/+/582799/

The most recent run says these two need updating:

# Run  npm install --save-dev [email protected]  to resolve 1 vulnerability
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ Low           │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ yargs-parser                                                 │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ stylelint-config-wikimedia [dev]                             │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ stylelint-config-wikimedia > stylelint > meow > yargs-parser │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://nodesecurity.io/advisories/1500                      │
└───────────────┴──────────────────────────────────────────────────────────────┘


# Run  npm update yargs --depth 2  to resolve 1 vulnerability
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ Low           │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ yargs-parser                                                 │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ @wdio/cli [dev]                                              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ @wdio/cli > yargs > yargs-parser                             │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://nodesecurity.io/advisories/1500                      │
└───────────────┴──────────────────────────────────────────────────────────────┘

Change 602169 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] bridge: Update dependencies to pick up security fixes

https://gerrit.wikimedia.org/r/602169

Change 602175 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] TR: Update dependencies to pick up security fixes

https://gerrit.wikimedia.org/r/602175

@Addshore Maybe I'm missing something obvious but looking at https://gerrit.wikimedia.org/r/c/integration/config/+/573235/3/jjb/job-templates.yaml it does only checks the main wikibase package.json and not the important ones like bridge, TR, termbox (honestly, running npm audit on these submodules exploded majestically)

Is that intended?

Change 602177 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Update wikimedia-stylint-config

https://gerrit.wikimedia.org/r/602177

I see there’s already a home-brewed solution to this problem but nevertheless I'd like to leave something here:
https://dependabot.com/ - this is a bot that regularly checks for updates in a repo's dependencies. If any are found it updates them and creates a pull request. It works for various languages.
This is how it looks (on a private project of mine) - https://github.com/tzhelyazkova/schema-form-prototype/pull/3
It should be possible to be configured to work with gerrit, here's the core lib - https://github.com/dependabot/dependabot-script

In T244001#6201253, @Tonina_Zhelyazkova_WMDE wrote:

I see there’s already a home-brewed solution to this problem but nevertheless I'd like to leave something here:

Maybe I'm missing something obvious, but that supposed to work in Github only and the ones here are in gerrit (can we run dependabot in gerrit? I'd be really surprised).

Change 602169 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] bridge: Update dependencies to pick up security fixes

https://gerrit.wikimedia.org/r/602169

Maybe I'm missing something obvious, but that supposed to work in Github only and the ones here are in gerrit (can we run dependabot in gerrit? I'd be really surprised).

It is now native to GitHub, but I cannot think of a reason this bot cannot work with gerrit.
The last link in my message describes how it works in more detail, and it mentions how to set the bot up with GitLab.

Again, I just wanted to let y'all know a solution already exists, and it does more than what our thing does. So we might want to weigh in the pros and cons of both.
I did not spend a lot of time researching dependabot, so I don't know how to integrate it with gerrit.

For all I care this could be a Beer and Cake project :)

So, there is a bot for gerrit that does various upgrades for extensions etc.
And this now works for Wikibase https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/605191
It also reports vulnerable packages to https://libraryupgrader2.wmflabs.org/vulns/npm

Having this work on multiple package.json files in a single repo is T228527: Support nested package.json files

We might skip this in story time, as the phrasing of the description now sounds like we just need to make a decision as a team and then write a ticket for doing it (or make a cal event)

If the approach boils down to "Patch Tuesday", I'd assign this to management. We've had taken a first stab on introducing this as a "team activity" in the late 2020. Initial implementation failed due to the gallore of other "special" things happening in Deecember, but kickstarting it again would be easy and fairly non controversial.
If there is no technical/code work involved, I would not see it as a campsite task no more.

If there is no technical/code work involved, I would not see it as a campsite task no more.

I think if at estimation the technical solution is small it may be worth it (for more promptly noticing these things).
I think the manual solution would need to be facilitated with a list (such as the one we discussed in a retro today).
@WMDE-leszek I was going to propose asking the wider team to see if we as a whole would be happy with just an email reminder to do it.

In terms of reducing developer toil I think more time would likely be saved for implementing the technical solution rather than having someone manually run npm audit in 20+ places or so ever week or so

Oh, I missed the possible alternative approach, apologies (got too excited this is talking about the "team day" only as the comment mentioned a calendar event)

As I will not be able to join the Planning/Story time, my input to the discussion here: If the "implementing the technical solution" is preferred what is not yet defined is what to do in cases where trivial automated fix are not sufficient (basically the essential reason to actually have really people look into those deps)

In terms of the technical solution, this would only be to serve us a notification, not to do the actual fix.
So upon notification then someone would need to go look at the result of the audit and determine what libraries need updating etc.

what is not yet defined is what to do in cases where trivial automated fix are not sufficient

Thus this bit would be done on a case by case basis.

The jenkins job for the root of wikibase has actually been running for over a year now!
https://integration.wikimedia.org/ci/job/wikibase-daily-npm-audit-daily-node10-npmaudit-docker
I actually can't see that it has failed all year (probably because library upgrader also runs on the root of the repo updating the easy things that can be)

FWIW lerna-audit now (v1.3.1) does not show the problems anymore which we encountered earlier (e.g.) when using it to tend to monorepos.

This was picked up on campsite for now with the AC of:

Acceptance criteria ⛺✨ :
Know all of the places that we have package.json files that need to be audited (and their current state in temrs of audit being run)

A dump of our repos to consider can be found at https://phabricator.wikimedia.org/P14784

Change 691265 had a related patch set uploaded (by Addshore; author: Addshore):

[mediawiki/extensions/WikibaseManifest@master] Dependabot -> Gerrit, Github Action

https://gerrit.wikimedia.org/r/691265

I looked once again at the draft attempt at creating an action for this (was https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/626687)
It is now reimagined in the change above which has been tested.

My bot account (Addbot) is currently used.
We can trial this on the WikibaseManifest extension, see how it goes, and maybe roll this out more later.

I should also look at making this easily reusable and maintainable.
https://docs.github.com/en/actions/creating-actions/creating-a-composite-run-steps-action

Change 602175 abandoned by Ladsgroup:

[mediawiki/extensions/Wikibase@master] TR: Update dependencies to pick up security fixes

Reason:

Already done.

https://gerrit.wikimedia.org/r/602175

Change 602177 abandoned by Ladsgroup:

[mediawiki/extensions/Wikibase@master] Update wikimedia-stylint-config

Reason:

Already done.

https://gerrit.wikimedia.org/r/602177

Change 691265 merged by jenkins-bot:

[mediawiki/extensions/WikibaseManifest@master] Dependabot -> Gerrit, Github Action

https://gerrit.wikimedia.org/r/691265

Change 572828 abandoned by Tarrow:

[mediawiki/extensions/Wikibase@master] TR: Add as dev dependency of Wikibase

Reason:

https://gerrit.wikimedia.org/r/572828