Svoboda | Graniru | BBC Russia | Golosameriki | Facebook

BBC Russian

Maniphest T350732

workflow_utils conda gitlab CI templates broken
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Ottomata
	Nov 7 2023, 8:00 PM

Tags

Referenced Files

None

Subscribers

Description

(Filed by @MunizaA)

openjdk 11.0.21 introduced a change which requires java.security file to be present in order to access java.security.Security. In conda_setup_script, this change breaks the post-installation script for ca-certificates-java when it is installed as a dependency of openjdk-11-jdk.

This is blocking several gitlab CI pipelines that use workflow_utils gitlab CI templates.

Explicitly installing ca-certificates-java before installing openjdk-11-jdk fixes this issue.

There are other crufty things that need fixed too in order to merge this fix.

miniconda checksum hashes URL has been moved and changed format. This is used by fixture_conda_base_env when testing some conda.py related functions.
Pin fsspec version to 2022.3.0 until workflow_utils test code is fixed to work with newer version

https://gitlab.wikimedia.org/repos/data-engineering/workflow_utils/-/merge_requests/39

Related Objects

Mentioned In: T347884: [Event Platform] mw-page-content-change-enrich should not retry on badrevids if no replica lag

Event Timeline

Ottomata created this task.Nov 7 2023, 8:00 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 7 2023, 8:00 PM

Ottomata claimed this task.Nov 7 2023, 8:00 PM

Ottomata mentioned this in T347884: [Event Platform] mw-page-content-change-enrich should not retry on badrevids if no replica lag.

Ottomata edited projects, added Data Engineering and Event Platform Team (Sprint 4); removed Data Engineering and Event Platform Team.

otto updated https://gitlab.wikimedia.org/repos/data-engineering/workflow_utils/-/merge_requests/39

fixes for conda ci templates

otto merged https://gitlab.wikimedia.org/repos/data-engineering/workflow_utils/-/merge_requests/39

fixes for conda ci templates

otto opened https://gitlab.wikimedia.org/repos/data-engineering/mediawiki-event-enrichment/-/merge_requests/75

To verify that a fix for T350732, use ci template from workflow_utils main

otto merged https://gitlab.wikimedia.org/repos/data-engineering/mediawiki-event-enrichment/-/merge_requests/75

To verify that a fix for T350732, use ci template from workflow_utils main

For completeness, I reproed this issue on mediawiki-content-dumps: https://gitlab.wikimedia.org/repos/data-engineering/dumps/mediawiki-content-dump/-/jobs/159004

otto opened https://gitlab.wikimedia.org/repos/data-engineering/mediawiki-event-enrichment/-/merge_requests/76

.gitlab-ci.yml - use workflow_utils v0.14.0 CI templates

Ottomata moved this task from Next Up to Done on the Data Engineering and Event Platform Team (Sprint 4) board.Nov 7 2023, 8:35 PM

xcollazo opened https://gitlab.wikimedia.org/repos/data-engineering/dumps/mediawiki-content-dump/-/merge_requests/20

Update file .gitlab-ci.yml

xcollazo opened https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/37

Update .gitlab-ci.yml

xcollazo merged https://gitlab.wikimedia.org/repos/data-engineering/dumps/mediawiki-content-dump/-/merge_requests/20

Update file .gitlab-ci.yml

xcollazo merged https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/37

Update .gitlab-ci.yml

otto merged https://gitlab.wikimedia.org/repos/data-engineering/mediawiki-event-enrichment/-/merge_requests/76

.gitlab-ci.yml - use workflow_utils v0.14.0 CI templates

lbowmaker closed this task as Resolved.Nov 10 2023, 1:50 PM