Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NoMongo: Add allUsers Query #3186

Open
rahulch07 opened this issue Feb 9, 2025 · 10 comments
Open

NoMongo: Add allUsers Query #3186

rahulch07 opened this issue Feb 9, 2025 · 10 comments
Assignees

Comments

@rahulch07
Copy link
Contributor

rahulch07 commented Feb 9, 2025

Is your feature request related to a problem? Please describe.
A query is needed to fetch all the users from the database for administrative purpose. This is been discused in this conversation.

Describe the solution you'd like
Make a query to fetch all the users from postgres database.

Additional context

Related issue Talawa-Admin #3527

Image

Potential internship candidates

Please read this if you are planning to apply for a Palisadoes Foundation internship

Copy link

github-actions bot commented Feb 9, 2025

Congratulations on making your first Issue! 🎊 If you haven't already, check out our Contributing Guidelines and Issue Reporting Guidelines to ensure that you are following our guidelines for contributing and making issues.

@github-actions github-actions bot added the unapproved Unapproved for Pull Request label Feb 9, 2025
@rahulch07
Copy link
Contributor Author

please assign this is related to feature I am already working on #3527

@palisadoes
Copy link
Contributor

  1. Fetching all users will be a resource drain. We have seen poor performance in the Events calendar because of over fetching
  2. How can you do this so that it lazy loads the data?

@rahulch07
Copy link
Contributor Author

@palisadoes

Solution:

  1. Cursor-Based Pagination:
  • Fetching all users at once can lead to performance issues, especially with large datasets.
  • To mitigate this, I will implement cursor-based pagination, which is the recommended approach for GraphQL.
  • This will allow data to be loaded in smaller chunks rather than all at once, improving efficiency and reducing strain on the database.
  1. Efficient Data Fetching:
  • Instead of retrieving all users in a single request, the initial query will fetch a limited subset (e.g., the first 10 users).
  • Additional users will be loaded dynamically as needed (e.g., when the user clicks "Next Page").
  • This approach ensures smooth performance while preventing unnecessary data over-fetching.

please take a look at this video where i have demonstrated this approach at api using graphiql

allUsersQueryVideo.webm

@palisadoes
Copy link
Contributor

Ok. Go ahead

@palisadoes palisadoes removed the unapproved Unapproved for Pull Request label Feb 10, 2025
@palisadoes
Copy link
Contributor

Are you still working on this?

@rahulch07
Copy link
Contributor Author

@palisadoes

Apologies for the delay in completing this issue. While implementing the pagination feature, I discovered a bug in the backward pagination logic that required additional debugging and fixing, which took some extra time.

@rahulch07
Copy link
Contributor Author

rahulch07 commented Feb 16, 2025

@palisadoes @xoldd PTAL

The issue was with the way we handled the previousPage check. We were fetching limit + 1 items from the database to determine if a previous page existed. If the length of the fetched data was equal to limit + 1, it indicated that a previous page was available. After setting the previousPage flag to true, we were supposed to remove the extra fetched element (i.e., the last item), but the existing logic was incorrectly removing the first element instead.

This caused backward pagination to skip the last item and instead return elements beyond the intended range. As a result, the pagination would break—forward pagination worked fine, but backward pagination was returning incorrect sequences (as shown in the image).

I’ve now fixed this issue, which should ensure that backward pagination returns the correct results. That’s why it took a bit longer than expected.

//defaultGraphQLConnection.ts
Image

Image

Forward pagination works fine. But for backward pagination it will skip few elements. Here the first page should have 4 members when navigated backwards but it has only two.

Pagination.Bug.webm

Fixed Backward Pagination

Pagination.Bug.Fixed.webm

@palisadoes
Copy link
Contributor

  1. Thanks. I never tested this in the past. It's good to remember to do it.
  2. Is this a universal issue in the API? If so we should create a separate issue to fix it.

@rahulch07
Copy link
Contributor Author

  1. Thanks. I never tested this in the past. It's good to remember to do it.
  2. Is this a universal issue in the API? If so we should create a separate issue to fix it.

Yes. This is a universal issue as the bug is present in defaultGraphQLConnection.ts. Creating issue for this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Backlog
Development

No branches or pull requests

2 participants