Answer Europeana @Hmangas Hugo Manguinhas Platform Services Product Manager API

I have asked others for opinions in Telegram Wikidata

• Wikidata can be accessed in different ways
  • SPARQL federation see Nobel prize
  • Video how you easy from a SPARQL query generates code in Python, JavaScript, PHP, Java, Perl, Ruby....
  • when I updated Wikidata I did a Python program that did webscrape Europeana and then I updated wikidata in batches using a tool Quickstatement see my latest batches
  • a new approach I have seen by @Andrawaag ( http://www.micelio.be/ ) is using Wikibase and create a staging area that is just updated by Europeana and Wikidata is then populated from that.... then the Wikibase installation is write protected and you have versioning and SPARQL with federation....
    • example diseases.semscape.org
    • video at 10:10 about the state of art of Wikibase by Lydia 2019-10 were she also mention Wikibase as a tool for synchronize data, at 13:50 she speaks about Structured Data in Commons that use Wikibase (another video about SDC, I guess the big challenge will be when we do dataroundtrip for linked data in pictures)

@David_Haskiya_WMSE @Jopparn do you have any suggestions for dataroundtrip process and tools of entity management between an open system like Wikidata and a source like Europeana

Other tools:

• Wikidata change stream
  • API:RecentChanges
  • SPARQL items connected to Europeana that has been merged the last week, see also deleted items I found T240738
    • script checking WD <-> Sameas Europeana

• Wikibase data-import
  • part of a site in progress learningwikibase.com
• wbstack.com free service to set up a new Wikibase with 3 click
  • WBstack presentation at Wikidata conf 2019 lightening talk time 19:00
  • blogpost WBStack – November review
  • blogpost WBStack 2020 Update 1
  • Active telegram group "Wikibase community"
  • product manager for Wikibase is @Samantha_Alipio_WMDE User:Samantha_Alipio_(WMDE)
    • Wikidata Wikibase Development plan
    • Project Wikibase - federated_properties - about
    • OCLC evaluation of Wikibase / video Lessons from representing library metadata in OCLC research’s Linked Data Wikibase prototype

see also EPIC T202530: [Epic] Feedback processes and tools for data-providers

ping @Mohammed_Sadat_WMDE (webpage) to get you in the loop if using Wikibase could be a possibility as a staging area between Wikidata and Europeana. In version one it is for managing entities of people (Europeana agents)

• Scenarios: create, merge, delete
  1. Europeana creates a new agent that already exist in Wikidata but don't have Property 7704 set
  2. Europeana deletes an agent that exist in Wikidata and has Property 7704 set
  3. Wikidata merges objects that exist in Europeana and has same as Wikidata
  4. Wikidata deletes an objects that exist in Europeana and has same as Wikidata
  5. Europeana creates an new Agent that doesnt exist in Wikidata

If this is the way forward using Wikibase I don't know but I like the concept more and more having a "staging area" using Wikibase were e.g. Europeana can upload the data, we can compare it and also comment individual records using Wikibase standard functionality with version history....

Why not have WikibaseP7704 that is a staging for WD property P7704 ....

Todays solution

• inventing the wheel for every new dataprovider doesnt scale ( we have > 4000 5962 external properties ) well with quality is my guess
• my experience with OCLC, Nobelprize, ... they doesnt have version history as we have in a Wiki or possibility to discuss items... and ping people...
• by having a staging area outside Wikidata we will also have federation and easy compare the delivered data to Wikibase what is in Wikidata --> we can easier detect vandalization of Wikidata by comparing what is delivered...
  • I mentioned in Wikidata Berlin 2017 (video at 42 min) a vision that in the Wikipedia articles infobox we should indicate if data displayed is the same as a "trusted" source delivered maybe this is step 1 in a solution like that .

Linked data needs Linked people is my guess I raised this problem at SWIBL18

Next step: can be

• that we get facts from a trusted source to a staging area either by the provider or that we webscrape --> we can easy compare Wikidata facts with the staging area
• external sources that move in the same way as Structured Data on Commons (also Wikibase) supporting linked data in pictures also start comparing the metadata with Wikicommon and use a staging area for differences between external object xxx and Wikibase object EntityData/M63516379 in Wikicommon (more about dataroundtrip of entities in pictures uploaded to Wikicommons.)

Example of Quickstatement that can be used to loaded a Wikibase instance / Wikidata, I guess when we have defined the entity management change process we need to set some flags if its a new item or old one....

the lines below says Q759804 has P7704 agent/base/34971. S813 is optional but sets a timestamp as source

• manual Quickstatement

Q759804 P7704 "agent/base/34971" S813 +2019-12-1225T00:00:00Z/11
Q206820 P7704 "agent/base/61829" S813 +2019-12-1225T00:00:00Z/11
Q730008 P7704 "agent/base/73049" S813 +2019-12-1225T00:00:00Z/11

Another alternative used by OCLC in the evaluation of WIkibase is Pywikibot see task T251625: Purge Europeana entries used for purging WD objects

Try to get feedback from WikiTree developer that has done data synch with Wikitree for the last 3 years and have a framework for improving the quality in WikiTree based on data from Wikidata / Findagrave.and also logic defined in > 200 rules. They also have a very active community running weekends cleaning sessions, creating videos etc... the site was created by Chris Whitten to order his family tree before a wedding.... today it has 23 million profiles....

• reports run weekly to check quality
• Wikidata property 2949 = 136 511 connected persons, property created 20160707
• example how differences is presented for the users
• example Wikidata check "Suggestion 541 Wikidata - Clue for Father"

• about Wikitree

Using Wikibase as a staging area is an interesting idea, but it is an idea that seems to only make sense after WMDE’s work to enable Wikidata/Wikibase federation is completed. This idea should be revisited then, and we can follow up on the topic with Europeana at that point.

@Salgo60 As a reminder, please do not share contents (or screenshots) of emails/private conversations in tickets as this creates a privacy issue. We cannot be sure of the impact that sharing these messages publicly may have on the original sender.

@Mohammed_Sadat_WMDE I think we have different level of staging and the federation possibility is I guess useful later. My feeling is that this could be something like mix-and-match 2.0 were we also get

• communication - using talk pages
• handling duplicates / mismatches / data round trips
• with > 4000 external identifiers we cant invent the wheel everytime as we also have 14000 active editors we need an easy way for everyone involved to communicate better than today with external sources and also understand open issues and status. Listen to Lydia 2017 video at 42 min were she agree we need a an echosystem maybe this is a building block for organisation not having public work boards/trackable issues / visibility of Sprints delivered ....
• getting a controlled change stream and possibilities to handle/document mismatches better than today . In Sweden we nearly never get helpdesk ID from organisations when we have an issue and its very seldom easy to track the status of a request or subscribe to an issue. Doing linked data without the correct tools and "connected people" will never work. Good video link about how loosely coupled networks needs to have trusts to work together and one piece is having tools...

My understanding is that Europeana plans a sprint thinking about how they would like to work together with an open platform like Wikidata and I can try to write down some user cases with my experience importing data for 10+ WD properties...

I think finding good patterns working together between loosely coupled platforms will be a key for achieving good things.... Europeana with a very loosely coupled network of I guess more than 6000 museums is a great test for trying in the first phase just getting the European <-> Wikidata <-> 205 language Wikipedia working together...

As Europeana works direction linked data video I think they look into using Wikidata as a possibility for museums to reference WD when they upload data to Europeana see MET-2410...

• Good video from Future Learn about Collaboration and Trust between loosely coupled networks

Discussion "Wikidata:Requests for comment/Handling of stored IDs after they've been deleted or redirected in the external database"

I informed about this thought with a staging area

Status 2020 jun 13: no feedback Europeana and I havnt pushed it and I dont know were in the Europeana backlog integration with Wikidata is

More feedback that WIkidata <-> Europeana data is not in sync see Property_talk:P7704#Linkrot

"Any update on this, I have been getting no found errors for quite sometime with this property"