Skip to content

Commit

Permalink
Move transformation and indexing of external records (RPB-248)
Browse files Browse the repository at this point in the history
  • Loading branch information
fsteeg committed Mar 4, 2025
1 parent 08c5fa4 commit 7bed920
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions transformAndIndex.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,6 @@ cat conf/output/output-strapi-external.ndjson | grep '"nur BiblioVino"\|"RPB und
rm conf/output/bulk/bulk-*.ndjson
# Transform:
sbt "runMain rpb.ETL conf/rpb-titel-to-lobid.flux index=$INDEX"
sbt "runMain rpb.ETL conf/rpb-titel-to-lobid-external.flux input=$RPB_INPUT url=$RPB_URL secret=$RPB_SECRET"
sbt "runMain rpb.ETL conf/rpb-titel-to-lobid-external.flux input=$VINO_INPUT url=$VINO_URL secret=$RPB_SECRET"

# Index to Elasticsearch:
unset http_proxy # for posting to weywot3
Expand Down Expand Up @@ -62,3 +60,7 @@ curl -X POST "weywot3:9200/_aliases?pretty" -H 'Content-Type: application/json'
]
}
'

# Transform and index external records (after index switch, they use the RPB and BiblioVino instances):
sbt "runMain rpb.ETL conf/rpb-titel-to-lobid-external.flux input=$RPB_INPUT url=$RPB_URL secret=$RPB_SECRET"
sbt "runMain rpb.ETL conf/rpb-titel-to-lobid-external.flux input=$VINO_INPUT url=$VINO_URL secret=$RPB_SECRET"

0 comments on commit 7bed920

Please sign in to comment.