Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 3.0.0 #227

Merged
merged 61 commits into from
Mar 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
0966adb
REC-269 Store errors with function names in report
nikosT Sep 12, 2023
445a77b
Merge pull request #184 from nikosT/rec269
kaggis Sep 12, 2023
5e83276
REC-266 Keep one provider per resource item
nikosT Sep 12, 2023
89dd6d6
Merge pull request #185 from nikosT/rec266
kaggis Sep 12, 2023
298215c
REC-272 Update UI to display warnings on calculation errors
kaggis Sep 12, 2023
0514d06
Merge pull request #186 from kaggis/ui-handle-errors
nikosT Sep 13, 2023
86f8d69
REC-175 Update get_catalog to use exact query parameter for marketpla…
kaggis Sep 19, 2023
cde3913
Merge pull request #187 from kaggis/get-catalog-use-exact-param
nikosT Sep 19, 2023
2084156
REC-277 convert np.nan to None types
nikosT Sep 20, 2023
d56537d
Merge pull request #188 from nikosT/fixnone
kaggis Sep 20, 2023
6b4c76c
REC-279 Update healthcheck to include recommendations stream check
kaggis Sep 20, 2023
748e872
Merge pull request #189 from kaggis/update-healthcheck
nikosT Sep 20, 2023
550fcba
Add badge in README
nikosT Sep 26, 2023
7ff0a48
Merge pull request #190 from ARGOeu/nikosT-patch-1
themiszamani Oct 5, 2023
a9dd078
Update urllib version based on alert
nikosT Oct 5, 2023
71720fb
Merge pull request #192 from nikosT/devel
themiszamani Oct 5, 2023
f531b0c
REC-242 Users are not longer retrieved. Users are generated based on …
nikosT Oct 12, 2023
64246fe
Merge pull request #195 from nikosT/rec242
kaggis Oct 13, 2023
f33d8a5
Typo in item views definition
nikosT Oct 14, 2023
da2da74
Merge pull request #196 from nikosT/devel
themiszamani Oct 14, 2023
a564aa0
REC-264 Convert top5 ordered to top5 viewed, fix in item views and no…
nikosT Oct 6, 2023
5b107be
Merge pull request #194 from nikosT/rec264
themiszamani Oct 18, 2023
75f5427
REC-281 Bump urllib3 to 1.26.18 and fix pyarrow version
nikosT Oct 26, 2023
24ec5e9
Merge pull request #201 from nikosT/devel
themiszamani Oct 26, 2023
4b578ed
REC-283 Bump werkzeug from 2.2.3 to 3.0.1 and flask fixes
nikosT Oct 26, 2023
1f33c3e
Merge pull request #202 from nikosT/rec283
themiszamani Oct 26, 2023
fc5b462
REC-284 Add Total User Actions and Item Views in metrics and UI
nikosT Nov 14, 2023
e37fa4c
Merge pull request #203 from nikosT/devel
kaggis Nov 14, 2023
4f3ce47
REC-263 Add timestamp to resources and add tag in metrics
nikosT Nov 23, 2023
dbbe5ae
Merge pull request #204 from nikosT/rec263
kaggis Nov 29, 2023
89f8a54
REC-286 Monitor for entries in Datastore
nikosT Nov 30, 2023
e2f9b22
REC-290 Bugfix in get_catalog concerning services and data_sources ca…
nikosT Dec 1, 2023
e33cb36
Merge pull request #206 from nikosT/rec286
themiszamani Dec 1, 2023
44d34ff
Merge pull request #207 from nikosT/rec290
kaggis Dec 5, 2023
a942c08
REC-292 Add e-mail support for the RS Monitor
nikosT Dec 8, 2023
7c66185
REC-293 Add support for capacity report in CSV and plot
nikosT Dec 8, 2023
2724939
Merge pull request #209 from nikosT/rec293
kaggis Dec 8, 2023
8914e39
Fix in e-mail subject
nikosT Dec 8, 2023
d53e059
Merge pull request #210 from nikosT/email_fix
themiszamani Dec 8, 2023
fd7b430
REC-297 Add support to ignore timestamp when necessary
nikosT Jan 3, 2024
186af8a
Merge pull request #211 from nikosT/rec297
kaggis Jan 8, 2024
40056ec
REC-298 fix bad type of resource_id in recommendations
nikosT Jan 8, 2024
1fa3dea
Merge pull request #212 from nikosT/rec298
kaggis Jan 9, 2024
f407a8a
Handover docs added
nikosT Jan 10, 2024
64a558c
Merge pull request #214 from nikosT/devel
themiszamani Jan 10, 2024
a982821
REC-296 fix links in top 5 items
nikosT Jan 12, 2024
ed0547c
REC-299 filter resource types according to configuration
nikosT Jan 12, 2024
3f42ccc
Updating versions
nikosT Jan 15, 2024
c6667fb
Merge pull request #217 from nikosT/rec296
themiszamani Jan 16, 2024
40ea5de
Merge pull request #218 from nikosT/rec299
themiszamani Jan 16, 2024
9edf497
Merge pull request #219 from nikosT/update_versions
themiszamani Jan 16, 2024
36eebec
REC-300 Add report description in /website and README
nikosT Jan 19, 2024
84b5576
Merge pull request #220 from nikosT/devel
themiszamani Jan 22, 2024
2c6b631
REC-289 Add Athena support in rs-stream.py
nikosT Feb 21, 2024
b363ee8
Merge pull request #221 from nikosT/rec289
nikosT Feb 22, 2024
e80e57c
AO-910 Add documentation cc info
kaggis Mar 4, 2024
085a4ab
Merge pull request #222 from kaggis/doc-cc
themiszamani Mar 4, 2024
2822511
Update LICENSE
kkoumantaros Mar 4, 2024
c23a0ec
Merge pull request #223 from ARGOeu/LICENCE-patch-1
nikosT Mar 4, 2024
4bd1a6c
AO-909 Update to docusaurus 3.x
kaggis Mar 11, 2024
677980a
Merge pull request #224 from kaggis/docu-update
themiszamani Mar 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Jenkinsfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pipeline {
agent {
docker {
image 'node:buster'
image 'node:18-buster'
}
}
options {
Expand Down Expand Up @@ -82,4 +82,4 @@ pipeline {
}
}
}
}
}
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright [yyyy] [name of copyright owner]
Copyright 2022 National Infrastructures for Research and Technology - GRNET S.A.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
36 changes: 35 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# Recommender Metrics Framework
A framework for generating statistics, metrics, KPIs, and graphs for Recommender Systems
<a href="https://eu.badgr.com/public/assertions/YWmU9yPvQxumUgd1vnPPSw">
<img src="https://api.eu.badgr.io/public/assertions/YWmU9yPvQxumUgd1vnPPSw/image" width="4%"/>
</a>

<p align="center">
<a href="https://github.com/ARGOeu/eosc-recommender-metrics/blob/ec222c3090892f33056086b8d30c18f713da519d/website/docs/static/img/flow.png">
Expand Down Expand Up @@ -55,8 +58,13 @@ A framework for generating statistics, metrics, KPIs, and graphs for Recommender
./rsmetrics.py -p athena # same procedure as the first one for 'athena' provider
```

10. A typical `rsmetrics.py` command for a monthly report, would be:
```bash
./rsmetrics.py -p provider -s $(date +"%Y-%m-01") -e $(date +"%Y-%m-%d") -t "$(date +"%B %Y")"
```

### Usage of the Streaming System
10. Run from terminal `./rs-stream.py` in order to listen to the stream for new data, process them, and store them in the `Datastore`, concerning that particular provider:
11. Run from terminal `./rs-stream.py` in order to listen to the stream for new data, process them, and store them in the `Datastore`, concerning that particular provider:
```bash
./rs-stream.py -a username:password -q host:port -t user_actions -d ""mongodb://localhost:27017/datastore"" -p provider_name
```
Expand Down Expand Up @@ -157,4 +165,30 @@ You can override this by editing the `.env` file inside the `/webservice` folder

_Tested with python 3.9_

#### Monitor for entries in the MongoDB collections
A typical example that counts the documents found in `user_actions`, `recommendations`, and `resources` for 1 day ago would be:
```bash
./monitor.py -d "mongodb://localhost:27017/rsmetrics" -s "$(date -u -d '1 day ago' '+%Y-%m-%d')" -e "$(date -u '+%Y-%m-%d')"
```

E-mail send over SMTP for the above example:
```bash
./monitor.py -d "mongodb://localhost:27017/rsmetrics" -s "$(date -u -d '1 day ago' '+%Y-%m-%d')" -e "$(date -u '+%Y-%m-%d')" --email "smtp://server:port" sender@domain recipient1@domain recipient2@domain

```

#### Export Capacity information for entries in the MongoDB collections
A typical example that counts the documents found in `user_actions`, `recommendations`, and `resources` for 1 year ago would be:
```bash
./monitor.py -d "mongodb://localhost:27017/rsmetrics" -s "$(date -u -d '1 day ago' '+%Y-%m-%d')" -e "$(date -u '+%Y-%m-%d')" --capacity
```
which will return results in CSV format of `year,month,user_actions,recommendations`

Additionally, capacity can be plotted:

```bash
./monitor.py -d "mongodb://localhost:27017/rsmetrics" -s "$(date -u -d '1 day ago' '+%Y-%m-%d')" -e "$(date -u '+%Y-%m-%d')" --capacity --plot
```

## Deployment docs
Installation and configuration documents can be found [here](docs).
6 changes: 3 additions & 3 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
providers:
- name: marketplace_rs
db: "mongodb://localhost:27017/recommender_dev"
- name: athena
- name: content-based engine
db: "mongodb://localhost:27017/athena_dev"
- name: online_engine
db: ""
Expand All @@ -17,12 +17,12 @@ service:
# Use the EOSC-Marketplace webpage
# to retrieve resources and associate the page_id and the service_id
store: './page_map' # or null
service_list_url: 'https://localhost/replace/with/remote/example'
service_list_url: 'https://search.marketplace.eosc-portal.eu/api/web/search-results'
# if true it keeps only published, otherwise all
# this has an effect in exporting when from is set to 'source'
# and also in metrics calculations where service is considered
published: true
category:
athena: [service]
content-based engine: [service]
marketplace_rs: [service]
online_engine: [training, data_source]
7 changes: 7 additions & 0 deletions docs/ChangeLog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Licence

<! --- SPDX-License-Identifier: CC-BY-4.0 -- >

## Change Log

- Initial version of this documentation.
Loading
Loading