-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add workflow basis to fetch HEBIS Data RPB-225 #105
Open
TobiasNx
wants to merge
15
commits into
main
Choose a base branch
from
rpb-225-fetchAndTransformHebis
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 13 commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
3153738
Add first workflow to fetch HEBIS Data RPB-225
TobiasNx 39d4908
Reuse lobid-transformation for hebis data RPD-225
TobiasNx d52a7b6
Adjust transformation and add tests for hebis RPB-225
TobiasNx c3d159e
Adjust transformation for fetching hebis records
TobiasNx 6127f70
Add workflow to fetch data for comparison from rpb
TobiasNx eaa1d0e
Bump Metafacture Version
TobiasNx 5fdfe75
Update to current lobid transformation #225
TobiasNx 9fca285
Add holdings from 924 #225
TobiasNx dcfa0c3
Merge remote-tracking branch 'origin/main' into rpb-225-fetchAndTrans…
TobiasNx c366a29
Update github actions build
TobiasNx 9f9e982
Add a suggestion for the workflow RPB-225
TobiasNx 07fe5d8
Rename folder for hebis transformation RPB-225
TobiasNx deedcd5
Outcomment describedBy transformation RPB-225
TobiasNx bb13c82
Update transformation to current lobid fix RPB-225
TobiasNx f73b85b
Add SRU Links to describedBy RPB-225
TobiasNx File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| default IDS = FLUX_DIR + "hebisTestIds.txt"; | ||
| createEndTime = "0"; //needed for lobid transformation | ||
|
|
||
| IDS | ||
| | open-file | ||
| | as-lines | ||
| | match(pattern="^(.*)$", replacement="http://sru.hebis.de/sru/DB=2.1?query=pica.ppn+%3D+%22$1%22&version=1.1&operation=searchRetrieve&stylesheet=http%3A%2F%2Fsru.hebis.de%2Fsru%2F%3Fxsl%3DsearchRetrieveResponse&recordSchema=marc21&maximumRecords=10&startRecord=1&recordPacking=xml&sortKeys=LST_Y%2Cpica%2C0%2C%2C") | ||
| | write(FLUX_DIR + "hebisSruLinks.txt") | ||
| ; | ||
|
|
||
| FLUX_DIR + "hebisSruLinks.txt" | ||
| | open-file | ||
| | as-lines | ||
|
|
||
|
|
||
| // the hebis workflow could start with the following and enter the the hebisId in strapi you start a workflow with the hebisId as parameter. | ||
| //hebisId | ||
| //| template("http://sru.hebis.de/sru/DB=2.1?query=pica.ppn+%3D+%22${o}%22&version=1.1&operation=searchRetrieve&stylesheet=http%3A%2F%2Fsru.hebis.de%2Fsru%2F%3Fxsl%3DsearchRetrieveResponse&recordSchema=marc21&maximumRecords=10&startRecord=1&recordPacking=xml&sortKeys=LST_Y%2Cpica%2C0%2C%2C") | ||
| | open-http(accept="application/xml") | ||
| | decode-xml | ||
| | handle-marcxml | ||
| | fix(FLUX_DIR + "hebisMarc2lobid-transformation/marcToLobid.fix",*) | ||
| | batch-reset(batchsize="1") | ||
| | encode-json(prettyPrinting="true") | ||
| | write(FLUX_DIR + "output/test-hebis-to-lobid-output-${i}.json") | ||
| ; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| default IDS = FLUX_DIR + "rpbEntriesOfHebisRecords.txt"; | ||
|
|
||
| IDS | ||
| | open-file | ||
| | as-lines | ||
| | match(pattern="^(.*)$", replacement="https://rpb.lbz-rlp.de/$1?format=json") | ||
| | open-http(accept="application/xml") | ||
| | as-records | ||
| | decode-json(recordPath="member") | ||
| | batch-reset(batchsize="1") | ||
| | encode-json(prettyPrinting="true") | ||
| | write(FLUX_DIR + "lobid-transformation/comparisonRpbRecords/rpb-hebis-records-${i}.json") | ||
| ; | ||
|
|
93 changes: 93 additions & 0 deletions
93
conf/hebisMarc2lobid-transformation/comparisonRpbRecords/rpb-hebis-records-0.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| { | ||
| "extent" : "62 Seiten : Illustrationen", | ||
| "contribution" : [ { | ||
| "agent" : { | ||
| "id" : "http://rpb.lobid.org/sw/00Gn24m0017186a", | ||
| "label" : "Caltapanides, Anke", | ||
| "type" : [ "Person" ] | ||
| }, | ||
| "role" : { | ||
| "id" : "http://id.loc.gov/vocabulary/relators/edt", | ||
| "label" : "Herausgeber/in" | ||
| }, | ||
| "type" : [ "Contribution" ] | ||
| }, { | ||
| "agent" : { | ||
| "id" : "https://d-nb.info/gnd/123750354X", | ||
| "label" : "Ingelheim am Rhein / Stadtverwaltung", | ||
| "type" : [ "CorporateBody" ] | ||
| }, | ||
| "role" : { | ||
| "id" : "http://id.loc.gov/vocabulary/relators/isb", | ||
| "label" : "Herausgebendes Organ" | ||
| }, | ||
| "type" : [ "Contribution" ] | ||
| } ], | ||
| "rpbId" : "036t240017", | ||
| "subject" : [ { | ||
| "id" : "http://purl.org/lobid/rpb#n520100", | ||
| "label" : "Sozialwesen allgemein", | ||
| "source" : { | ||
| "id" : "http://purl.org/lobid/rpb", | ||
| "label" : "Systematik der Rheinland-Pfälzischen Bibliographie" | ||
| }, | ||
| "type" : [ "Concept" ] | ||
| }, { | ||
| "id" : "http://purl.org/lobid/rpb#n843020", | ||
| "label" : "Öffentliches Gebäude", | ||
| "source" : { | ||
| "id" : "http://purl.org/lobid/rpb", | ||
| "label" : "Systematik der Rheinland-Pfälzischen Bibliographie" | ||
| }, | ||
| "type" : [ "Concept" ] | ||
| }, { | ||
| "componentList" : [ { | ||
| "id" : "https://d-nb.info/gnd/4026954-1", | ||
| "label" : "Ingelheim am Rhein", | ||
| "source" : { | ||
| "id" : "https://d-nb.info/gnd/7749153-1", | ||
| "label" : "Gemeinsame Normdatei (GND)" | ||
| } | ||
| }, { | ||
| "id" : "https://d-nb.info/gnd/7607842-5", | ||
| "label" : "Mehrgenerationenhaus", | ||
| "source" : { | ||
| "id" : "https://d-nb.info/gnd/7749153-1", | ||
| "label" : "Gemeinsame Normdatei (GND)" | ||
| } | ||
| }, { | ||
| "id" : "http://rpb.lobid.org/sw/00Gn18k3167156a", | ||
| "label" : "Geschichte 2009-2019", | ||
| "source" : { | ||
| "id" : "http://rpb.lobid.org/sw", | ||
| "label" : "RPB-Sachsystematik" | ||
| } | ||
| } ], | ||
| "label" : "Ingelheim am Rhein | Mehrgenerationenhaus | Geschichte 2009-2019", | ||
| "type" : [ "ComplexSubject" ] | ||
| } ], | ||
| "publication" : [ { | ||
| "publishedBy" : [ "Stadtverwaltung Ingelheim am Rhein" ], | ||
| "location" : [ "Ingelheim am Rhein" ], | ||
| "type" : [ "PublicationEvent" ] | ||
| } ], | ||
| "responsibilityStatement" : [ "Redaktionsteam: Anke Caltapanides [und 3 weitere] ; Herausgeber: Stadtverwaltung Ingelheim am Rhein" ], | ||
| "inCollection" : [ { | ||
| "id" : "http://lobid.org/resources/HT013494180#!", | ||
| "label" : "Rheinland-Pfälzische Bibliographie", | ||
| "type" : [ "Collection" ] | ||
| } ], | ||
| "id" : "https://lobid.org/resources/036t240017", | ||
| "type" : [ "BibliographicResource", "Book" ], | ||
| "title" : "Wie wir wurden was wir sind - 10-jähriges Jubiläum des Mehrgenerationenhauses Ingelheim", | ||
| "spatial" : [ { | ||
| "id" : "https://rpb.lobid.org/spatial#n33900030", | ||
| "label" : "Ingelheim am Rhein, große kreisangeh. Stadt", | ||
| "source" : { | ||
| "id" : "https://rpb.lobid.org/spatial", | ||
| "label" : "RPB-Raumsystematik" | ||
| }, | ||
| "type" : [ "Concept" ] | ||
| } ], | ||
| "@context" : "http://lobid.org/resources/context.jsonld" | ||
| } |
62 changes: 62 additions & 0 deletions
62
conf/hebisMarc2lobid-transformation/comparisonRpbRecords/rpb-hebis-records-1.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| { | ||
| "extent" : "1 gefaltetes Blatt (16 Seiten) : Illustrationen", | ||
| "note" : [ "Parallele Sprachausgabe: The ancient cathedral St. John" ], | ||
| "rpbId" : "036t240110", | ||
| "subject" : [ { | ||
| "id" : "http://purl.org/lobid/rpb#n844030", | ||
| "label" : "Sakralbau", | ||
| "source" : { | ||
| "id" : "http://purl.org/lobid/rpb", | ||
| "label" : "Systematik der Rheinland-Pfälzischen Bibliographie" | ||
| }, | ||
| "type" : [ "Concept" ] | ||
| }, { | ||
| "componentList" : [ { | ||
| "id" : "https://d-nb.info/gnd/4369072-5", | ||
| "label" : "Sankt Johannis / Mainz", | ||
| "source" : { | ||
| "id" : "https://d-nb.info/gnd/7749153-1", | ||
| "label" : "Gemeinsame Normdatei (GND)" | ||
| } | ||
| } ], | ||
| "label" : "Sankt Johannis / Mainz", | ||
| "type" : [ "ComplexSubject" ] | ||
| } ], | ||
| "responsibilityStatement" : [ "Evangelisches Dekanat Mainz" ], | ||
| "alternativeTitle" : [ "The ancient cathedral St. John" ], | ||
| "inCollection" : [ { | ||
| "id" : "http://lobid.org/resources/HT013494180#!", | ||
| "label" : "Rheinland-Pfälzische Bibliographie", | ||
| "type" : [ "Collection" ] | ||
| } ], | ||
| "type" : [ "BibliographicResource", "Book" ], | ||
| "title" : "Alter Dom St. Johannis : 1500 Jahre Geschichte im Herzen von Mainz", | ||
| "@context" : "http://lobid.org/resources/context.jsonld", | ||
| "contribution" : [ { | ||
| "agent" : { | ||
| "id" : "https://d-nb.info/gnd/2009806-6", | ||
| "label" : "Evangelische Kirche in Hessen und Nassau", | ||
| "type" : [ "CorporateBody" ] | ||
| }, | ||
| "role" : { | ||
| "id" : "http://id.loc.gov/vocabulary/relators/isb", | ||
| "label" : "Herausgebendes Organ" | ||
| }, | ||
| "type" : [ "Contribution" ] | ||
| } ], | ||
| "publication" : [ { | ||
| "publishedBy" : [ "Evangelisches Dekanat Mainz" ], | ||
| "location" : [ "Mainz" ], | ||
| "type" : [ "PublicationEvent" ] | ||
| } ], | ||
| "id" : "https://lobid.org/resources/036t240110", | ||
| "spatial" : [ { | ||
| "id" : "https://rpb.lobid.org/spatial#n31500000", | ||
| "label" : "Mainz, Kreisfreie Stadt", | ||
| "source" : { | ||
| "id" : "https://rpb.lobid.org/spatial", | ||
| "label" : "RPB-Raumsystematik" | ||
| }, | ||
| "type" : [ "Concept" ] | ||
| } ] | ||
| } |
91 changes: 91 additions & 0 deletions
91
conf/hebisMarc2lobid-transformation/comparisonRpbRecords/rpb-hebis-records-10.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| { | ||
| "extent" : "127 S. : zahlr. Ill., Kt.", | ||
| "rpbId" : "036t000008", | ||
| "subject" : [ { | ||
| "id" : "http://purl.org/lobid/rpb#n102050", | ||
| "label" : "Ortsbeschreibung", | ||
| "source" : { | ||
| "id" : "http://purl.org/lobid/rpb", | ||
| "label" : "Systematik der Rheinland-Pfälzischen Bibliographie" | ||
| }, | ||
| "type" : [ "Concept" ] | ||
| }, { | ||
| "componentList" : [ { | ||
| "id" : "http://rpb.lobid.org/sw/z64", | ||
| "label" : "Geschichte 1945-", | ||
| "source" : { | ||
| "id" : "http://rpb.lobid.org/sw", | ||
| "label" : "RPB-Sachsystematik" | ||
| } | ||
| }, { | ||
| "id" : "https://d-nb.info/gnd/4155569-7", | ||
| "label" : "Führer", | ||
| "source" : { | ||
| "id" : "https://d-nb.info/gnd/7749153-1", | ||
| "label" : "Gemeinsame Normdatei (GND)" | ||
| } | ||
| } ], | ||
| "label" : "Geschichte 1945- | Führer", | ||
| "type" : [ "ComplexSubject" ] | ||
| } ], | ||
| "isbn" : [ "3-87439-488-3" ], | ||
| "responsibilityStatement" : [ "[Konzept und Texte: Frank Herda ; Georg Schuler ; Ellen Vest]" ], | ||
| "inCollection" : [ { | ||
| "id" : "http://lobid.org/resources/HT013494180#!", | ||
| "label" : "Rheinland-Pfälzische Bibliographie", | ||
| "type" : [ "Collection" ] | ||
| } ], | ||
| "type" : [ "BibliographicResource", "Book" ], | ||
| "title" : "Mainz transparent : 2000 Jahre einer Stadt ; [Stadtbegleiter]", | ||
| "@context" : "http://lobid.org/resources/context.jsonld", | ||
| "contribution" : [ { | ||
| "agent" : { | ||
| "id" : "http://rpb.lobid.org/sw/00Pn00m0019c", | ||
| "label" : "Herda, Frank", | ||
| "type" : [ "Person" ] | ||
| }, | ||
| "role" : { | ||
| "id" : "http://id.loc.gov/vocabulary/relators/aut", | ||
| "label" : "Verfasser/in" | ||
| }, | ||
| "type" : [ "Contribution" ] | ||
| }, { | ||
| "agent" : { | ||
| "id" : "http://rpb.lobid.org/sw/00Pn00m0019a", | ||
| "label" : "Schuler, Georg", | ||
| "type" : [ "Person" ] | ||
| }, | ||
| "role" : { | ||
| "id" : "http://id.loc.gov/vocabulary/relators/aut", | ||
| "label" : "Verfasser/in" | ||
| }, | ||
| "type" : [ "Contribution" ] | ||
| }, { | ||
| "agent" : { | ||
| "id" : "http://rpb.lobid.org/sw/00Pn00m0019b", | ||
| "label" : "Vest, Ellen", | ||
| "type" : [ "Person" ] | ||
| }, | ||
| "role" : { | ||
| "id" : "http://id.loc.gov/vocabulary/relators/aut", | ||
| "label" : "Verfasser/in" | ||
| }, | ||
| "type" : [ "Contribution" ] | ||
| } ], | ||
| "publication" : [ { | ||
| "publishedBy" : [ "Schmidt" ], | ||
| "location" : [ "Mainz" ], | ||
| "type" : [ "PublicationEvent" ], | ||
| "startDate" : "2000" | ||
| } ], | ||
| "id" : "https://lobid.org/resources/036t000008", | ||
| "spatial" : [ { | ||
| "id" : "https://rpb.lobid.org/spatial#n31500000", | ||
| "label" : "Mainz, Kreisfreie Stadt", | ||
| "source" : { | ||
| "id" : "https://rpb.lobid.org/spatial", | ||
| "label" : "RPB-Raumsystematik" | ||
| }, | ||
| "type" : [ "Concept" ] | ||
| } ] | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@acka47 files named
conf/hebisMarc2lobid-transformation/comparisonRpbRecords/rpb-hebis-records-n.jsonare for only for comparison they are records from the productive rpb.